TIP #412: DYNAMIC LOCALE CHANGING FOR MSGCAT WITH ON-DEMAND FILE LOAD ======================================================================= Version: $Revision: 1.2 $ Author: Harald Oehlmann State: Draft Type: Project Tcl-Version: 8.6 Vote: Pending Created: Tuesday, 27 March 2012 URL: http://tcl.activestate.com/cgi-bin/tct/tip/412.html Post-History: Obsoletes: TIP #399 ------------------------------------------------------------------------- ABSTRACT ========== This TIP adds dynamic locale switching capabilities to the *msgcat* package. RATIONALE =========== DYNAMIC LOCALE SWITCHING -------------------------- Within a multi-language application like a web-server, one may change the locale quite frequently, for example if users with different locales are requesting pages. Unfortunately, this does not fit well with the model adopted by the *msgcat* package, which assumes that all code follows this sequence: 1. Set locale list: *mclocale* /locale/ 2. Load language files with other package load: *mcload* /msg-folder/ 3. Translate strings: *mc* /key args.../ Note that if the locale should be changed after other packages are loaded, one must restart at step 2. This requires reloading all packages which is mostly not practical. The aim of this TIP is to extend the package by dynamic locale change capabilities. msgcat will reload any missing message catalog files of all currently loaded packages on a locale change. In addition, any package may register to get informed to a locale change. Other packages may do changes to reflect the locale change like rebuilding the GUI. This TIP compares to [TIP #399] that the package is able to load message catalog files on demand, e.g. specially on a locale change. PACKAGE LOCALE ---------------- If the clock command gets called with the argument "-locale", the locale is changed using *msgcat::mclocale*. After processing, the initial value is restored. The package keeps track, which locales where already used and calls *msgcat::mcload* for any new locale. The locale is restored after processing. This is an implementation of dynamic locales but conflicts with the new features described above. Other packages may be informed to change the locale and may trigger expensive operations like a rebuild of the GUI. In consequence, each package may define a package locale which is independent of the default locale. SPECIFICATION =============== PACKAGE EQUALS CLIENT NAMESPACE --------------------------------- A *client package* is a package which uses msgcat. A unique namespace is required for each client package. Within msgcat, namespace and package is always connected. Up to now, the msgcat package used this namespace as an identifier to store the catalog data of a certain package. This is now extended to additional properties which are stored for a package. PACKAGE LOCALE ---------------- A package locale may be used by a package instead the default locale *msgcat::mclocale*. A package locale may be set or unset by a package. DEFAULT AND PACKAGE STATE --------------------------- All State values (like the current locale) are available, once as default and once per package, if a package sets a *package locale*. The used naming is: * default state: valid for all packages which do not set a package locale. * package state: only valid for one package if it has set a package locale. The following state values are present as default state and may be set individually per package. * The *locale* state value is the value from *mclocale*. * The *preferences* state value is the value from *mcpreferences*. * The *loadedlocales* state value is a list of currently loaded locales. In particular, an *mcload* was issued for those locales and they were not cleared in-between. DEFAULT STATE A new command supports operations on the default state: *msgcat::mcloadedlocales* /subcommand/ The default configuration may be get set by: locale: set by the command *mclocale* preferences: set by the command *mcpreferences* loadedlocales: set through the action of the new command *msgcat::mcloadedlocales load* The default configuration may be set by: All those state values are set by a change of the default locale by *mclocale locale*. This command does the following operations: * Build and set the preferences. * Load all message catalog files for all packages without a package locale for all locales, which are in the new preferences but not in *loadedlocales*. * Invoke the callbacks of all packages which have registered for a locale change callback. After a change of the default locale, the user may decide to clear all now unneeded loaded locales. This is possible using the new command: *msgcat::mcloadedlocales clear* The list of currently loaded locales is set to *mcpreferences* and all message catalog keys of packages without a package locale set and with locales not in *mcpreferences* are unset. PACKAGE CONFIGURATION ----------------------- The package configuration of the calling package may be changed using the following new command: *msgcat::mcpackagelocale* /subcommand/ ?/locale/? The parameter /locale/ may only be specified with the subcommand *set*. Available subcommands are: SUBCOMMAND "SET" Set or change the package locale. The global state values are copied, if there were no package locale set before. The package locale is changed to the optional given new package locale. SUBCOMMAND "GET" Return the package locale or the default locale, if no package locale set. SUBCOMMAND "PREFERENCES" Return the package preferences or the default preferences, if no package locale set. SUBCOMMAND "LOADED" The list of locales loaded for this package is returned. SUBCOMMAND "ISSET" Returns true, if a package locale is set. SUBCOMMAND "UNSET" Unset the package locale and use the default state for the package. Load all message catalog files of the package for locales, which were not present in the package *loadedlocales* list and are present in the default list. SUBCOMMAND "CLEAR" Set the current loaded locales list of the package to *mcpreferences* and unset all message catalog keys of the package with locales not included in the package *preferences*. It is an error to call this subcommand without a package locale set. PACKAGE CONFIGURATION OPTIONS ------------------------------- Each package may have a set of configuration options set to invoke certain actions. They may be retrieved or changed with the following new command: *msgcat::mcpackageconfig* /subcommand option/ ?/value/? Available subcommands are: get: Get the current value of the option or an error if not set. isset: Returns true if option is set. set: Set the given value to the option. May have additional consequences and return values as described in the option section. unset: Unset the option. Available options are: PACKAGE OPTION "MCFOLDER" This is the message folder of the package. This option is set by *mcload* and by the subcommand *set*. Both are identical and both return the number of loaded message catalog files. Setting or changing this value will load all locales contained in the preferences valid for the package. This implies also to invoke any set *loadcmd* (see below). Unsetting this value will disable message file load for the package. If the locale valid for this package changes, this value is used to eventually load message catalog files. Message catalog files are always sourced in the namespace of the package registering the value. PACKAGE OPTION "LOADCMD" This callback is invoked before a set of message catalog files are loaded for the package which has this property set. This callback may be used to do any preparation work for message file load or to get the message data from another source like a data base. In this case, no message files are used (*mcfolder* is unset). See chapter *callback invocation* below. The parameter list appended to this callback is the list of locales to load. If this callback is changed, it is called with the preferences valid for the package. PACKAGE OPTION "CHANGECMD" This callback is invoked when a default local change was performed. Its purpose is to allow a package to update any dependency on the default locale like showing the GUI in another language. Tk may be extended to register to this callback and to invoke a virtual event. See the *callback invocation* section below. The parameter list appended to this callback is *mcpreferences*. All registered packages are invoked in no particular order. PACKAGE OPTION "UNKNOWNCMD" Use a package locale *mcunknown* procedure instead of the standard version supplied by the msgcat package (*msgcat::mcunknown*). The called procedure must return the formatted message which will finally be returned by *msgcat::mc*. A generic unknown handler is used if set to the empty string. This consists in returning the key if no arguments are given. With given arguments, format is used to process the arguments. See chapter *callback invocation* below. The appended arguments are identical to mcunknown. CALLBACK INVOCATION Callbacks are invoked under the following conditions: * the callback command is set, * the command is not the empty string, * the registration namespace exists. Any error within the callback stops the operation which invoked the callback. This might be surprising, as the error might be in another package. TEST IF MESSAGE KEY IS SET ---------------------------- Message catalog keys may be expensive to calculate and thus may be set on demand. The following new procedure returns false, if *mc* would call *mcunknown* for a key: *msgcat::mcexists* /src/ There are two options, to limit the key search to just the current namespace (don't search in parent namespaces) and just the current locale (don't search the preferences but the first item): *msgcat::mcexists* ?*-exactnamespace*? ?/-exactlocale/? /src/ FORGET PACKAGE ---------------- A package may clear all its keys and state using the new command: *msgcat::mcforgetpackage* LOCALE AND PREFERENCES FORMAT ------------------------------- Locales set by *mcset* may eventually not correspond to the current preferences, as the preferences are treated as follows: * put to lower case, * remove any multiple "_" and any "_" at the beginning or at the end of the locale. It is proposed, that: * the locale and the first preferences element is always identical to the lowercase passed locale, * any multiple "_" are seen as one separator. Example: preferences of locale "sy__cyrl_win" * current preferences: "sy_cyrl_win sy_cyrl sy" * proposed preferences: "sy__cyrl_win sy__cyrl sy". Alternatively, all locales may normalized using the upper algorithm, which felt heavy in computation with little gain. EXAMPLE USAGE =============== EXAMPLE FROM TIP #399 ----------------------- Imagine an application which supports the current user language and French, German and English. An external package *tp* is used. The package uses *msgcat* and installs itself during the *package require tp* call: package require msgcat msgcat::mcload [file join [file dirname [info script]] msgs] An implementation of the application with the current msgcat 1.5.0 would require the following initialization sequence: package require msgcat package require np and the following code to change the locale to French: package forget np msgcat::mclocale fr package require np Using the extension of this TIP, one may load as usual: package require msgcat package require np and to change to french locale: msgcat::mclocale fr The first time, a locale is required, all corresponding message files of all packages which use msgcat get loaded. This might be a heavy operation. If a locale is reactivated (and the message catalog data was not cleared), it is a quick operation. Without this TIP, it is computational expensive (if possible, as many packages are not reloadable or a reload may disturb current processing, e.g., by forcing the closing of sockets, etc.). CHANGE WITH NO NEED TO COME BACK ---------------------------------- If it is certain that a locale is changed and the then obsolete data is of no use, one may clear unused message catalog items: msgcat::mclocale fr msgcat::mcloadedlocale clear USE A CALLBACK TO BE NOTIFIED ABOUT A LOCALE CHANGE ----------------------------------------------------- Packages which display a GUI may update their widgets when the locale changes. To register to a callback, use: namespace eval gui { msgcat::mcpackageconfig changecmd updateGUI proc updateGui args { puts "New locale is '[lindex $args 0]'." } } % msgcat::mclocale fr fr % New locale is 'fr'. TO USE ANOTHER LOCALE SOURCE THAN MESSAGE CATALOG FILES --------------------------------------------------------- If locales (or additional locales) are contained in another source like a data base, a package may use the load callback and not /mcload/: namespace eval db { msgcat::mcpackageconfig loadcmd loadMessages msgcat::mcconfig loadedpackages\ [concat [msgcat::mcconfig loadedpackages] [namespace current]] proc loadMessages args { foreach locale $args { if {[LocaleInDB $locale]} { msgcat::mcmset $locale [GetLocaleList $locale] } } } } USE A PACKAGE LOCALE ---------------------- The reference implementation also contains a changed clock command which uses a package locale. Here are some sketches from the implementation. First, a package locale is initialized and the generic unknown function is activated: msgcat::mcpackagelocale set msgcat::mcpackageconfig unknowncmd "" If the user requires the week day in a certain locale, it is changed: clock format [clock seconds] -format %A -locale fr and the code: msgcat::mcpackagelocale set $locale return [lindex [msgcat::mc DAYS_OF_WEEK_FULL] $day] ### Returns "mercredi" Some message-catalog items are heavy in computation and thus are dynamically cached using: proc ::tcl::clock::LocalizeFormat { locale format } { set key FORMAT_$format if { [::msgcat::mcexists -exactlocale -exactnamespace $key] } { return [mc $key] } #...expensive computation of format clipped... mcset $locale $key $format return $format } REFERENCE IMPLEMENTATION ========================== See Tcl fossil tag *msgcat_dyn_locale* []. COMPATIBILITY =============== Imagined incompatibilities: * If packages call *mcload* multiple times with different folders, the data was currently appended. This is still the case, but only the last folder is used for any reload. The property *mcfolder* may be transformed to a list to cover this case. * The return value of *mcload* (file count) may be much higher as there may be loaded much more files. I suppose, this value is only used by the test suite to verify functionality and is not for big general use. * Message files may not be aware, that they may be loaded at any moment and not only after their own *mcload*. I suppose, this is the biggest issue but I think, there is no alternative. * Message files do not get reloaded any more, if a second *mcload* is issued with the same path argument. * Package which temporary change the default locale trigger any callback and may lead to user visible side effects. ISSUES ======== Known issues: * Packages might not be aware of a locale change and may buffer translations outside of *msgcat*. Packages should not buffer msgcat messages if they are used in a dynamic locale application (like tklib tooltip does for example). * The clock command currently has a small dynamic patch for msgcat implemented. This must be removed in favor to new msgcat features due to the temporarily change of the default locale. EXTENSIONS ============ * Expose the function to calculate the preference list from a given locale. * Load a message catalog file for a given locale without changing the default/package locale. * Methods *isloaded* to check if a locale is currently loaded. * Access message catalog with specified namespace, locale and search behavior. ALTERNATIVES ============== The alternative is the former [TIP #399], but that is problematic because the list of locales must be known before any package load. The additional complexity of this TIP is a justifiable trade-off against the greatly improved flexibility in the loading and locale selection order. COPYRIGHT =========== This document has been placed in the public domain. ------------------------------------------------------------------------- TIP AutoGenerator - written by Donal K. Fellows