TIP #414 Version 1.11: Add (back) Tcl_InitSubsystems as Public API

This is not necessarily the current version of this TIP.

Abstract

The ability to initialize just the lower level Tcl subsystems used to be part of the public API, now it is no longer exposed. This TIP proposes that it be re-exposed.

Rationale

Some parts of Tcl's API are useful in portable applications even without creating a Tcl interpreter; examples of this include Tcl_Alloc and (most of) the Tcl_DString-related functions. In order to use these functions correctly, the Tcl library must be initialized, yet the function for doing so - Tcl_InitSubsystems (currently TclInitSubsystems) - was removed from Tcl's API; using Tcl_FindExecutable instead feels incorrect as we're not seeking to make the name of the executable available to Tcl scripts.

However, exposing the current TclInitSubsystems has the limitation that there is no way to control exactly what is initialized: What will be the system encoding, what will be the registered name of the executable, what panic proc should be used when a panic occurs during initialization. Therefore, the function will get a flags parameter which controls in finer detail how the initialization is done. The default, 0, means that the initialization is done exactly the same as Tcl_FindExecutable(NULL). But other flags can be set to modify the initialization, and more flag values can be defined in the future. This TIP proposes 5 additional values for the flags value.

The need for the TCL_INIT_CREATE_??? flags is inspired by the current complication of the Tk_Main and Tcl_Main macros, and the internal functions Tcl_MainEx, Tcl_MainExW, Tk_MainEx and Tk_MainExW which have a lot of code in common. With help of the Tcl_InitSubsystems, those macros could be simplified, or replaced with a totally different approach in Tcl 9. How that is done, is out of scope for this TIP.

Proposed Change

A new function Tcl_InitSubsystems, similar to the internal TclInitSubsystems, should be exposed as alternative to Tcl_FindExecutable in Tcl's C API. This will not be a part of the Stub API; it is not intended to ever be used from an initialized stubbed environment, as it is meant to be used prior to the stub table being available. It has a variable number of arguments, the first of which is flags which controls the interpretation of the additional arguments. If flags is 0, then this function does exactly the same as Tcl_FindExecutable(NULL). The full signature is:

EXTERN Tcl_Interp * Tcl_InitSubsystems( int flags, ...);

If you supply the flag TCL_INIT_PANIC to Tcl_InitSubsystems, the function expects an additional argument, a custom panicProc, immediately following the flags argument. This is equivalent to calling Tcl_SetPanicProc immediately before Tcl_InitSubsystems, except that you possibly cannot do that yet if it requires an initialized stub table. Of course Tcl_SetPanicProc could be called immediately after Tcl_InitSubsystems, but then panics which could be produced by the initialization itself still use the default panic procedure.

If you supply one of the flags TCL_INIT_CREATE, TCL_INIT_CREATE_UTF8 or TCL_INIT_CREATE_UNICODE to Tcl_InitSubsystems (possibly in combination with TCL_INIT_PANIC), the function gets two additional parameters, argc and argv. Then a new Tcl interpreter will be created. If argc > 0 then the variables argc and argv will be set in this interpreter. The 3 variants assume a different encoding for the arguments, except for argvTIP #0 which is always assumed to be in the system encoding. If argc is 0, still argv must be provided (possibly as NULL value) as it might be used to set the executable name used in scripts.

If you supply the flag TCL_INIT_CUSTOM to Tcl_InitSubsystems, the function expects two additional arguments following the previously mentioned ones: ClientData and a custom proc. The proc will be executed just before the encodings are initialized, and it will be given two arguments: the Tcl interpreter and ClientData. Remember that encodings other than UTF-8, unicode and iso8859-1 are not yet available when the customProc is run, the system encoding is not determined, and no variables are set in the interpreter yet. This means that no libaries/packages can be found yet, it is typically only used for calling functions like Tcl_SetEncodingSearchPath and Tcl_FSRegister.

The return value of Tcl_InitSubsystems is, if one of the TCL_INIT_CREATE_?? flag values is used, the newly created interpreter. If you want to use the Tcl library without the need to create a Tcl Interpreter, Tcl_InitSubsystems still returns a pointer to a Tcl_Interp structure, but the only allowed use for this pointer is as input parameter of the Tcl_InitStubs function. For example:

Tcl_InitStubs(Tcl_InitSubSystems(0), NULL, 0);

This will initialize the Stub table without the need for a real Tcl Interpreter. Applications embedding Tcl can now be compiled using USE_TCL_STUBS=1, and linked with the Tcl stub library (in addition to the Tcl library). After the Tcl_InitStubs call, all other parts of Tcl API can be called normally, as long as they don't need a real Tcl interpreter.

Another new possibility, allowed by this approach, is linking the application with the Tcl Stub library, but load the Tcl shared library dynamically. For example:

Tcl_Interp *(*initSubSystems)(int, ...);
void *handle = dlopen("libtcl8.6.so", RTLD_NOW|RTLD_LOCAL);
initSubSystems = dlsym(handle, "Tcl_InitSubsystems");
Tcl_InitStubs(initSubSystems(0), NULL, 0);

This way, the application doesn't depend on a single Tcl version any more, but it could determine at runtime which Tcl version is available (8.6, 8.7, 9.0 ...) and load the preferred version at runtime.

Reference Implementation

A reference implementation is available in the initsubsystems2 branch. [1]

Copyright

This document has been placed in the public domain.

This is not necessarily the current version of this TIP.