TIP #181 Version 1.1: Add a [namespace unknown] Command

This is not necessarily the current version of this TIP.

Abstract

This TIP proposing adding a new namespace subcommand, unknown, which would register a per-namespace procedure for dealing with unknown commands.

Rationale

There is an occassional need within Tcl scripts to change the way in which command names are resolved. For instance, when implementing language constructs such as object systems, or some functional programming ideas (such as anonymous first-class functions). In the case of implementing an object system, you may want a command to be searched for in the class namespace and then in the namespaces of any super-classes (assuming you have implemented classes in terms of namespaces containing procs). When trying to do anonymous functions, a useful technique is auto-expansion of leading word if a command is not found ([1] is an example of this technique).

Furthermore, it is sometimes useful to create new behaviour for what happens when a command doesn't exist at all in the interpreter, for instance to implement custom auto-loading mechanisms.

At present, Tcl's command resolution procedure tries to find commands in the current namespace, and then the global namespace, before finally using the global ::unknown proc to deal with unknown commands. There are several drawbacks with this mechanism.

Firstly, it is not possible to override the command resolution process from Tcl, except by redefining the ::unknown procedure to perform your custom lookup. However, by the time this procedure is called, the current and global namespaces have been already been searched, and so you can only change the behaviour as a case of last resort. This is especially difficult if you are writing a package, as good style dictates that you shouldn't override the global ::unknown procedure without being explicitly asked to do so.

Secondly, as Tcl searches for a hard-coded fallback procedure name (::unknown), in order to override it's functionality you have to rename it and then install your own replacement - and the new version becomes the default fallback behaviour for the entire application. In the case of implementing custom auto-loading behaviour, you may only want to override the behaviour for your package, and not for the entire interpreter. Currently, the only way to do this is to define a new ::unknown procedure which does pattern matching on the command name it is passed.

Finally, if a package does override the ::unknown procedure it has to be careful to save the old handler, and then invoke it for commands which it is not interested in. This is an error-prone approach, and results in a cascade of procedure calls, often with each one only interested in a subset of the commands being searched for.

Related TIPs

There have been two previous attempts at modifying Tcl's command resolution process. TIP #52 proposed that the search order be changed to traverse the complete namespace hierachy from most specific namespace to the most general (the global namespace). This TIP was withdrawn as it was not backwards compatible. TIP #142 proposed a global variable which would hold a namespace search path. This TIP was also withdrawn as it does not allow different namespaces to have different search paths.

Proposed Change

This TIP proposes that the handling of unknown commands be done on a per-namespace basis through the introduction of an unknown subcommand of the namespace command.

namespace unknown ?namespaceName? commandName

The subcommand would accept either one or two arguments. If two arguments are given, then the first must be the fully qualified name of a namespace which the command is to operate on. If only one argument is given, then this defaults to the current namespace. The other argument (or only argument in the one argument case) is a command to execute if command lookup in the given namespace fails. The command will be concatenated with the full invocation line of the command being searched for (i.e. the command name and all arguments), and evaluated in the scope of the current namespace. The command name given must be able to be resolved without resorting to the unknown mechanism (i.e. it must either be a command in the current namespace, or be fully-qualified). If this cannot be done, an error will be generated.

The command resolution procedure would be altered from this:

1. Lookup command in current namespace.

2. If that fails, lookup command in global namespace.

3. If that fails, call global ::unknown procedure.

to this:

1. Lookup command in current namespace.

2. If that fails, call the unknown handler for the current namespace.

The default unknown handler for each namespace will look for the command in the global namespace and if that fails call the global unknown handler. The global namespace will have a default unknown handler called ::unknown. This means that by default, we have exactly the same mechanism that exists currently in Tcl. In order to change the mechanism for an individual namespace, you may register a new unknown handler for that namespace.

The calling of unknown handlers registered with namespace unknown would be identical to the current calling of the ::unknown procedure - the handler will be called with the command name and all of its arguments, as it was originally invoked.

Notes on Ensembles

TIP #112 (Namespaces are Ensembles are Commands) added some features which allow for flexible handling of unknown subcommands when using ensembles. It is worth noting here that this is not the same thing as handling unknown commands in general. For instance, consider the following code:

namespace eval foo {
   proc useful {args} { ... }

   namespace eval bar {
      proc carrot {arg1 args} {
         useful $arg1
      }
   }
}

This case cannot be easily solved with ensembles.

Examples

Here are a few examples of the proposed functionality, in order to illustrate how it would work in practice. Firstly, here is the example from the previous section implemented via a custom unknown handler:

namespace eval foo {
   proc useful {args} { ... }

   namespace eval bar {
      # Note that the namespace command must be fully qualified
      namespace unknown [list ::namespace eval ::foo]
      proc carrot {arg1 args} {
         useful $arg1
      }
   }
}

Here is a more complicated example, of an object system with public/protected methods, showing how 'namespace unknown and namespace ensemble can be used together:

namespace eval SuperClass {
   # Define our "public" methods
   namespace export hello
   # Proc to create a new "instance"
   proc create {name} {
      namespace ensemble create -name $name
   }

   # This proc is "public"
   proc hello {} { puts "Hello from SuperClass!" }

   # This proc is "protected" - can be called by subclasses, but not directly
   # via the ensemble
   proc SomePrivateMethod {args} { puts $args }
}
namespace eval SubClass {
   # Public methods
   namespace export do
   # Handler to delegate to superclass
   namespace unknown [list ::namespace eval ::SuperClass]

   proc create {name} {
      # Create superclass instance
      ::SuperClass create $name.super
      # Create our instance
      namespace ensemble create -name $name \
              -unknown [list dispatch $name.super]
   }

   # Used for handling direct calls to super class methods
   proc dispatch {super this method args} {
      $super $method {expand}$args
   }

   # Our public method
   proc do {arg1 args} {
      SomePrivateMethod $arg1
   }
}

SubClass::create foo
foo do a b c ;# -> prints "a"
foo hello    ;# -> prints "Hello from SuperClass!"
foo SomePrivateMethod ;# -> error...

Finally, here is an example of an unknown handler which does more complicated processing:

namespace eval foo {
   namespace unknown infix

   # Allow infix-style syntax
   proc infix {args} {
      if {[regexp (.+):$ [lindex $args 0] -> name]} {
         set args [lreplace $args 0 0 $name =]
      } ;# allow REBOL-style assignments (foo: bar; bar: 17+4)
      if {[lindex $args 1]=="="} {
         # maybe an assignment like "x = 3+4" ? (Blanks matter!)
         upvar [lindex $args 0] _x
         set rest [lrange $args 2 end]
         if {[llength [info commands [lindex $args 2]]]} {
            return [set _x [uplevel eval $rest]]
         }
         set _x $rest                ;# this should always work...
         catch {set _x [expr $rest]} ;# ...but maybe expr is happy
         return $_x
      } elseif {[regexp {^([^ ]+)\+\+$} $args -> vname]} {
         uplevel [list incr $vname]  ;# allow things lie "i++" ...
      } elseif {[regexp {^([^ ]+)--$} $args -> vname]} {
         uplevel [list incr $vname -1]         ;# ... or "j--"
      } elseif {[regexp {^[-+/\*\.0-9 ()]+$} $args]} {
         return [expr $args]         ;# pure expression? "(17+4)/3"
      } else {
         # Look in global namespace...
         namespace eval :: $args
      }
   }

   proc dostuff {} {
      v = ::info tclversion
      s = this is a string
      j = sqrt(2)*3
   }
}

This last example is taken from the Radical Language Modification page on the wiki[2].

Reference Implementation

A reference implementation is not yet available, but the author intends to write one soon.

Copyright

This document has been placed in the public domain.

This is not necessarily the current version of this TIP.