TIP #112 Version 2.18: Ensembles are Namespaces are Commands

This is not necessarily the current version of this TIP.

Abstract

This TIP proposes unifying the concept of ensembles (from [Incr Tcl]) with namespaces and commands. It also adds control of command rewriting to allow for more efficient support for object systems like Snit.

Rationale

Tcl's subcommand-style command collections (e.g. array, info, string, interp, etc.) are a very intuitive and popular way of structuring collections of related commands. However, it is quite awkward to write Tcl code that behaves that way. Users of [Incr Tcl] have access to ensembles which provide that, but it would be a very useful feature for many other uses too.

At the same time, it is becoming clear that many applications want to commonly refer to commands inside other namespaces directly (instead of through the [namespace import] mechanism) but the syntax for doing this is verbose and not as elegant as it might be.

I believe that the same solution can address these two problems in one go, and make the language stronger and more usable for it.

Furthermore, by giving the programmer control over the mapping from the ensemble subcommands to their implementing commands, we can build a simple class system on the cheap since we can, in effect, import commands from elsewhere into the ensemble. Indeed, by extending the mapping so that it allows the specification of not just the implementing command, but also some leading arguments to that command (similar to what you can do with the [interp alias] mechanism) this becomes a very powerful mechanism indeed.

Finally, a sophisticated extra capability is the addition of an unknown subcommand callback to allow the creator of the ensemble to specify a customized strategy for handling subcommands that are not recognized by the ensemble machinery. This allows an ensemble to be dynamically updated to include those subcommands that the user asks for, which is a potential route for doing things like using an ensemble as a wrapper round a Tk widget.

Proposed Change

I propose to add a new subcommand to the [namespace] command, ensemble, that creates and manipulates the ensemble command for a namespace. Each namespace may have any number of ensemble commands associated with it, with the default name of an ensemble command being the fully-qualified name of the namespace itself, though it will be legal to rename ensemble commands (anyone wanting to track such events should use the [trace] command.) Tcl will not create an ensemble command for any namespace by default.

The [namespace ensemble] command will have the following subcommands:

create

For creating a new ensemble for the current namespace. The command takes a list of option value pairs (as defined below) to set the ensemble up with. As stated above, the default command name is exactly the name of the namespace, but any other name may be supplied (via the -command option); if it does not start with a namespace separator, the name will be taken as being relative to the current namespace. If another command with the same name exists, it is deleted as part of the creation of the ensemble.

configure

For reading and writing the configuration of a particular ensemble. The command takes an argument specifying the name of the ensemble to configure, and then works with either no extra arguments (when it retrieves the entire ensemble configuration), one extra argument (the name of the option to retrieve), or a list of option value pairs to configure the ensemble with. This command returns an error if applied to anything that is not an ensemble.

exists

This subcommand (which takes a single argument) tests whether a command (with the given name) exists and is an ensemble. This command only returns an error if the wrong number of arguments are supplied.

The options available for creation and configuring are:

-export

This option (if non-empty) specifies a list of ensemble subcommands that the ensemble supports. It does not need to be sorted. Each command is mapped according to the dictionary in the -map option (if a non-empty map is present and the command has a mapping) or to the correspondingly named command (which is not required to exist at the time this is specified) in the context namespace.

-map

This option (if non-empty) specifies a dictionary that maps from ensemble subcommand names to lists of arguments to substitute into the ensemble invokation in place of the ensemble command name and the subcommand name. See the -export option for the meaning of this option when that option is also non-empty. If this option is empty and the -export option is empty too, the namespace will use the exported commands of the namespace as its command set, dynamically determining them (subject to cacheing) every time the ensemble is invoked. Note that the words in the dictionary values that give the commands to map to (as opposed to any arguments to them) are always resolved (if not absolute) relative to the namespace which is running [namespace create] or [namespace configure].

-prefix

This boolean option (which is on by default) controls whether unambiguous prefixes of ensemble commands are recognized as if they were the fully specified ensemble command names.

-unknown

This provides (when non-empty) a partial command to handle the case where an ensemble subcommand is not recognized and would otherwise generate an error. It is executed by concatenating on all the arguments to the ensemble command (including the ensemble command name, though it is the fully qualified name that is passed) and then invoking the completed command in the scope of the caller; if the script terminates normally, the ensemble engine reparses the subcommand and tries to dispatch it again (which is ideal for when the ensemble's configuration has been updated by the unknown subcommand handler.) The result of the the unknown script is parsed as a list, and all other kinds of result are treated as errors. When the result is an empty list, the ensemble command attempts to look up the original subcommand again and, if it is not found this time, an error will be generated just as if the -unknown handler was not there (i.e. for any particular invokation of an ensemble, its unknown handler will be called at most once.) When the result is a non-empty list, the words of that list are used to replace the ensemble command and subcommand, just as if they had been looked up in the -map, though the command name itself will be looked up in the context of the code calling the namespace (and hence should be fully specified normally.)

Where an empty -unknown handler is given (the default), the ensemble command will generate an error message based on the list of commands that the ensemble has defined (formatted similarly to the error message from Tcl_GetIndexFromObj()). This error will also be thrown when the subcommand is still not recognized during reparsing. It is also an error for an -unknown handler to delete its namespace.

Ensemble creation takes an extra option value pair:

-command

This option allows you to override what the name of the ensemble to create is.

Ensemble configuring allows the querying of an extra read-only option:

-namespace

This read-only option allows the retrieval of the name of the namespace that the ensemble was created in.

Given an ensemble command created by the above mechanism, calling the command will first of all match the subcommand to its implementing command (or command/argument list, as derived from the dictionary) in a manner that will be recognizably similar to that enforced by Tcl_GetIndexFromObj() (unless the -unknown option override this behaviour.) For details of how the ensemble determines what its subcommands are, please see the -export and -map options above. Then the ensemble command will rewrite the command and arguments so that the ensemble command and subcommand are replaced by the implementing command and any specified arguments, with the resulting word list being fed to Tcl_EvalObjv() for execution. Note that this does not increase the stack depth in terms of [uplevel], and that the implementing command may itself be an ensemble command.

Note that if the namespace for an ensemble is deleted, the ensemble will also be deleted though there is no lifetime constraint in the other direction (i.e. using [rename $theEnsemble {}] will not cause the namespace to vanish.) Obviously, any ensemble which has the global namespace backing it up will have the same natural lifespan as the hosting Tcl interpreter.

Examples

namespace eval carrot {
   namespace export foo bar potato
   namespace ensemble create     ;# Creates command ::carrot

   proc foo {} {puts 1}          ;# Exported
   proc bar {} {puts 2}          ;# Exported
   proc boo {} {puts 3}          ;# Not exported

   namespace eval turnip {       ;# Not exported
      namespace export alpha
      proc alpha {} {puts 4}     ;# Exported
      proc beta {} {puts 5}      ;# Not exported
      namespace ensemble create
   }

   namespace eval potato {       ;# Exported
      proc north {x} {puts 6,$x} ;# Not exported
      proc south {x} {puts 7,$x} ;# Not exported
      # Notice we resolve names locally, and Tcl metachars are not special
      namespace ensemble create -map {
         north {north [$meta[$chars}
      }
   }
}


carrot foo                       ;# Prints 1
carrot bar                       ;# Prints 2
carrot b                         ;# Also prints 2 ("boo" not exported)
carrot ?                         ;# ERROR: Alternatives "bar", "foo" and "potato"
carrot potato                    ;# ERROR: Missing argument
carrot potato ?                  ;# ERROR: Try "north" instead
carrot potato north              ;# Prints 6,[$meta[$chars
carrot turnip alpha              ;# ERROR: "turnip" not known
carrot::turnip alpha             ;# Prints 4
carrot::turnip::beta             ;# Prints 5

rename ::carrot::potato ::spud
spud north                       ;# Prints 6
spud south                       ;# ERROR: "south" not known
carrot potato north              ;# ERROR: No ::carrot::potato command
# Reconfigure spud; notice we get different name resolution now!
namespace ensemble configure spud -map {
   north {puts NORTH} south {puts SOUTH}
}
spud north                       ;# Prints NORTH
spud south                       ;# Prints SOUTH
namespace delete carrot
spud north                       ;# ERROR: spud command already deleted


namespace eval A {
   proc a args {puts A::a=>$args}
}
namespace eval B {
   proc b args {puts B::b=>$args}
}
# Create an ensemble in the global namespace
namespace ensemble create -command C -map {
   eg1 {::A::a foo bar}
   eg2 {::B::b 1 2 3}
   eg3 ::string
}
C eg1 spong                      ;# Prints A::a=>foo bar spong
C eg2 evil code {[exit]}         ;# Prints B::b=>1 2 3 evil code [exit]
C eg3 length qwertyuiop          ;# Returns 10


# An example demonstrating the use of -unknown to do delegation
# This uses an ensemble to add a subcommand to a frame
package require Tk
rename [frame .f] _impl.f
namespace ensemble create -command .f -unknown {delegate _impl.f} -map {
   flash {flashFrame _impl.f}
}
# General delegation framework handler
proc delegate {target ensemble subcmd args} {
   # Read the current map
   set map [namespace ensemble configure $ensemble -map]
   # Update it to include the new subcommand
   dict set map $subcmd [list $target $subcmd]
   # Install back into the ensemble
   namespace ensemble configure $ensemble -map $map
   # Result is empty string, so we reparse
   return {}
}
# Our new subcommand implementation
proc flash w {
   set bg [$w cget -background]
   foreach colour {black white black white black white} {
      $w configure -background $colour
      update idletasks
      after 150
   }
   $w configure -background $bg
}

Consequences

Many commands in both Tcl and Tk would benefit from leveraging this, and it would enable straight-forward implementations of things like TIP #65 in pure Tcl code. It would also make doing things like partial exposure of ensemble-like commands in safe interpreters much easier.

Sample Implementation

http://sf.net/tracker/?func=detail&aid=786509&group_id=10894&atid=310894

Copyright

This document has been placed in the public domain.

Acknowledgements

Thanks very much to Joe English, Don Porter and Kevin Kenny for their suggestions in the development of this TIP. Without them, it would have been a far worse suggestion. And thanks to Will Duquette for writing a piece of software (Snit) that would benefit immensely from pushing the ensemble stuff as hard as possible and then a bit. :^)

This is not necessarily the current version of this TIP.