This is not necessarily the current version of this TIP.This is not necessarily the current version of this TIP.
| TIP: | 86 |
| Title: | Saving [proc] File and Line Number Information in [source] |
| Version: | $Revision: 1.16 $ |
| Authors: |
Peter MacDonald <peter at browsex dot com> Peter MacDonald <peter at pdqi dot com> |
| State: | Draft |
| Type: | Project |
| Tcl-Version: | 8.5 |
| Vote: | Pending |
| Created: | Friday, 08 February 2002 |
This TIP proposes the addition to Tcl of source code file-name and line-numbering information, making it available at script execution time.
A new [trace execution] option enables Tcl to track line number and source file information associated with statements being executed and call a single callback. A new [info line] option provides access to line number information. As a result, it becomes a simple matter to implement a debugger for Tcl, in Tcl. Furthermore, the implementation also serves as example usage of the C interface, enabling similar capabilities at the lower level.
A simple Tcl debugger, tgdb, written in Tcl, and emulating gdb, is included with this Tip to demonstrate the use of this interface. tgdb supports breakpoints on lines/procs/vars/etc, single-stepping, up/down stack and evals. It is designed to work both as a commandline and a slave process (see Reference Implementation).
Finally, upon error within a procedure, the file path and absolute (as opposed to relative) line number are printed out when available, even in the case where called from an after or callback invocation. Aside from aiding the user in more easily locating and dealing with errors, the message is machine parseable For example: automatically bring the user into an editor at the offending line.
A new execution subcommand to the trace command.
trace execution target ?level?
This arranges for an execution trace to be setup for commands at nesting level or above, thereby providing a simple Tcl interface for tracing commands to say, implement a debugger. With no arguments, the current target is returned. If target is the empty string, the execution trace is removed. Unless the target argument is a valid output channel such as stdout, it is assumed to be a command string to be executed. When level is not specified, it defaults to 0, meaning trace all commands. For each traced command, the following data will be produced:
linenumber
The line number the instruction begins on.
filename
The fully normalized file name.
nestlevel
The nesting level of the command.
stacklevel
The stack call level as per [info level].
curnsproc
The current fully qualified namespace/function.
cmdname
The fully qualified command name of the command to be invoked.
command
The command line to be executed including arguments.
If target is a valid Tcl channel opened for output, the trace data is output to it with an appended newline. Otherwise target is presumed to be a valid Tcl command onto which is appended the above arguments before evaluation. For channels, any error on output causes the trace to be removed and a tcl error to be generated. For commands, any return from the command other than a normal return results in the command not being executed.
Second, a new line subcommand to info gives access to the file path and line number information. It takes subcommands of its own in turn:
info line current
the line number and file name of the current statement.
info line number proc ?line?
Get/set the line number for the start of proc. In get mode, returns the definition line number in the message body.
info line level number
Like the info level command, but returns the line number and file name from which the call at level number originates. For use with trace execution.
info line file ?proc ?file??
Get/set the sourced file path for proc. If proc is not specified, dump all known sourced file paths.
info line find file line
Given a file path and line number, return the proc name containing line number line. A new nonstatic procedure TclFindProcByLine() provides this function.
info line relativeerror ?bool?
Set to 1 to disable absolute line number and file path on a procedure error. This demotes procedure traceback errors to the same format as all other traceback errors, that is, using the relative the line number and file name.
These exhibit the following behavior:
What does this do when you redefine a proc?
You get the values from the latest definition.
What about when you use interp aliases?
You get an error, as it is not considered a proc.
And if proc itself gets redefined by someone's special debugger?
If the definition is not the result of a source, the file/line come back as an empty string.
Third, a new info subcommand return.
info return
When in trace execution mode, returns the saved last result of the previously executed command. Otherwise returns an empty string. Commands executed as part of a trace handler do not affect or change the saved last result.
Forth, an additional flag option debug to trace add variable
trace add variable {write debug}
Used in conjuction with read, write, and unset, this option allows a debugger to set read/write traces that do not trigger the execution trace.
Sourced file paths are stored per interp in a hash table. File/line numbering information is also stored in the Interp, Proc, After, and CallFrame structures. Newline counting/shifting code was added to proc, while, for, foreach, and if. All but the non-trivial code is active only when the new TRACE_LINE_NUMBERS interp flag is active, which is the case when using trace execution.
Most new variables within Interp are in the struct subfield sourceInfo of type Tcl_SourceInfo, which can be retrieved via the new Tcl_GetSourceInfo(interp) stubbed/public call.
The runtime impact to Tcl should be modest: a few 10's of kilobytes of memory, even for moderately large programs. Most of the space impact occurs in storing the file paths. A typical example from a large system:
100 sourced files * 100 bytes = 10K.
The other space overhead adds up to several words (8 bytes on a 32-bit platform) per defined procedure, plus an additional words in the Interp structure.
Runtime processing overhead should be negligible.
However, there have been no benchmarks done to validate these assertions.
http://pdqi.com/download/tclline-cvs.diff.gz - Patch against CVS head.
http://pdqi.com/download/tclline-8.4.6.diff.gz - Patch against Tcl 8.4.6
The patch against the CVS head is as of June 13/2004 and should be considered the definitive patch. The 8.4.6 patch is provided for convienience and may or may not be updated. These have been lightly tested against numerous small Tcl programs.
There is also an initial version of a debugger: tgdb.
http://pdqi.com/download/tgdb-2.0.tar.gz
tgdb emulates the basic commands of gdb (s, n, c, f, bt, break, info locals, etc). This newest version also supports watchpoints and display variables. With load and run commands added, tgdb should probably work even with emacs and ddd.
An additional package pdqi provides tdb, a GUI front-end to gdb, modified to also work with tgdb.
Build and store a line number table internally during parse?
Line number lookup via the source string. A simple way to implement this might be to lookup string against the codePtr->source+bestSrcOffset as returned by GetSrcInfoForPc().
Add special handling for eval. Cases like eval $str should eventually be changed to report a line number of 0 (or more likely the line number of the original statement) for all statements with any argument involving a sub-eval.
Possibly implement character offsets within a line.
A test has been added to the tests/trace.test. A utility trcline.tcl is provided that the test uses to provide some measure of the accuracy of the line number tracing.
Jeff Hobbs asked what about interp alias, etc.
Updated TIP to document cases
Jeff Hobbs notes filename storage is inefficient and finalization
Code changed to just increment ref count
TODO: What needs to be done for Tcl_Finalize?
Neil Madden/Stephen Trier comment on info subcommand names line, file and proc and possible future uses for line
Changed to a single subcommand line and use sub-sub commands.
Additional subsubcommands can easily be added.
Donal Fellows writes: Is there a way to do an equivalent of #line directives in C
we can now set line number etc of a proc. Is that enough?
Donald Porter notes that changing Tcl_Parse breaks binary compatibility
Move all parse variables to Interp and save/restore values on entry/exit to Tcl_EvalEx and TclCompileScript.
Donald Porter notes that the hash table should be per Interp
Code changed to move hash table to Interp.
Mo DeJong notes: file path should be used in place of file name
TIP updated to use path where appropriate
Mo DeJong suggests to maybe use TclpObjNormalizePath(fileName)
No action yet
Donal Fellows objects to no support for [proc]s in subevals and Andreas Kupries suggests defining a line number Tcl_Token type.
Add support for proc in subeval by addition to ResolvedCmdName
This is now fixed.
This document has been placed in the public domain.
tgdb' and pdqi'' have a BSD copyright by Peter MacDonald and PDQ Interfaces Inc.
This is not necessarily the current version of this TIP.