TIP #86 Version 1.11: Saving [proc] File and Line Number Information in [source]

This is not necessarily the current version of this TIP.

Abstract

This TIP proposes saving the file path and line number information for each user defined [proc] definition contained within a [source]d file, and providing a new subcommand [line] to [info] that can return this new information.

Rationale

With this change, upon error within a procedure, the file path and absolute (as opposed to relative) line number are printed out when available, even in the case where called from an after or callback invocation. Aside from aiding the user in more easily locating and dealing with errors, the message is machine parseable For example: automatically bring the user into an editor at the offending line.

Second, a simple/small debugger emulating gdb is being developed that can use this interface. (This debugger is not part of this TIP but has been used to validate the interface) It allows breakpoints on lines, single-stepping and displaying/locating source and even works as a slave under a GUI written in Tk. Ultimately, at some point this debugger might be included in the core distribution via a future TIP.

Specification

The new [line] subcommand to [info] gives access to the file path and line number information for a given PROC. Valid syntax is:

  info line number PROC ?LINE?
        Get/set the line number for PROC.  In get mode, returns the
        definition line number in the message body. 

  info line newlines PROC ?NEWLINES?
        Get/set the count of newlines in the body of PROC.  In
        set mode, changes the count of the number of
        newlines in the message body.  This provides a fairly speedy
        way of determining if a given line number is included in a
        specific proc.

  info line file ?PROC ?FILE??
        Get/set the sourced file path for PROC.  If PROC is not
        specified, dump all known sourced file paths.

  info line find FILE LINE
        Given a file path and line number, return the PROC name
        containing line number LINE.  A new nonstatic procedure
        TclFindProcByLine() provides this function.

  info line relativeerror ?BOOL?
 Set to 1 to disable absolute line number and file path
       on a procedure error.  This demotes procedure traceback errors
       to the same format as all other traceback errors, that is,
       using the relative the line number and file name.

These exhibit the following behavior:

• What does this do when you redefine a proc?

  You get the values from the latest definition.

• What about when you use interp aliases?

  You get an error, as it is not considered a proc.

• And if proc itself gets redefined by someone's special debugger?

  If the definition is not the result of a source, the file/line come back as an empty string.

Changes

Sourced file paths are stored per interp in a hash table. Line numbering information is saved in the Proc structure. Several line number variables are added to Interp to track newlines.

The bulk of the changes occur in generic/tclParse.c where the number of line feeds seen is tracked.

Overhead/Impact

The runtime footprint of Tcl should be modest: a few 10's of kilobytes, even for moderately large programs. Most of the space impact occurs in storing the file paths, but these are stored only once each for each file sourced, not per interp. A typical example from a large system:

  100 sourced files * 100 bytes = 10K.

The other space overhead adds up to 2 words (8 bytes on a 32-bit platform) per defined procedure, plus an additional several words in the Interp structure.

Runtime processing overhead should be negligible.

However, there have been no benchmarks done to validate these assertions.

Reference Implementation

http://browsex.com/tclline.diff.gz

This is a patch against tcl8.4a4. It has been tested lightly against small and large Tcl programs.

There is also an initial version of a debugger: tgdb.

http://browsex.com/tgdb.tgz

tgdb emulates some of the basic commands of gdb (s, n, c, f, bt, break and info locals). This newest version also supports watchpoints and display variables.

The tgdb strategy is to determine the line number by searching from the reported line number forward for the currently executing command. Although not foolproof, in practice this is almost always accurate.

• When in a function, search in the procedure body

• When at global level, search in the file source starting from reported line.

Also bundled is tdb, a GUI front-end to gdb, modified to also work with tgdb. You can test it with:

  make 
  ./tdb test.tcl 

Once load and run commands are added, tgdb should probably work even with emacs and ddd.

Comments and Feedback

Jeff Hobbs asked what about interp alias, etc.

• Updated TIP to document cases

Jeff Hobbs notes filename storage is inefficient and finalization

• Code changed to just increment ref count

• TODO: What needs to be done for Tcl_Finalize?

Neil Madden/Stephen Trier comment on info subcommand names line, file and proc and possible future uses for line

• Changed to a single subcommand line and use sub-sub commands.

• Additional subsubcommands can easily be added.

Donal Fellows writes: Is there a way to do an equivalent of #line directives in C

• we can now set line number etc of a proc. Is that enough?

Donald Porter notes that changing Tcl_Parse breaks binary compatibility

• Move all parse variables to Interp and save/restore values on entry/exit to Tcl_EvalEx and TclCompileScript.

Donald Porter notes that the hash table should be per Interp

• Code changed to move hash table to Interp.

Mo DeJong notes: file path should be used in place of file name

• TIP updated to use path where appropriate

Mo DeJong suggests to maybe use TclpObjNormalizePath(fileName)

• No action yet

Donal Fellows objects to no support for [proc]s in subevals and Andreas Kupries suggests defining a line number Tcl_Token type.

• Add support for [proc] in subeval by addition to ResolvedCmdName

Copyright

This document has been placed in the public domain.

This is not necessarily the current version of this TIP.