This is not necessarily the current version of this TIP.This is not necessarily the current version of this TIP.
| TIP: | 259 |
| Title: | Making 'exec' Optionally Binary Safe |
| Version: | $Revision: 1.1 $ |
| Author: | Andreas Leitgeb <avl at logic dot at> |
| State: | Draft |
| Type: | Project |
| Tcl-Version: | 8.5 |
| Vote: | Pending |
| Created: | Monday, 12 December 2005 |
A new option shall be added to the command exec, that allows the user to specify that input redirected from immediate data (using <<) and/or the data received from the external command shall not undergo any translation within Tcl.
External programs may expect binary data, or write out binary data, or neither or even both. Whether a program reads/writes binary data or platform-encoded data is generally specific to the particular program and known by the programmer who intends to exec it from a Tcl script.
For example, a hexdump-utility expectably reads binary data and outputs text.
Problem 1: For passing string-data to external programs, exec now behaves arguably incorrect, because it does not pass through \0-bytes.
Problem 2: For returning the result of external programs, exec applies translations based on system-default encoding, which is OK in most cases, except, of course, for programs that output binary data.
Problem 1 is actually a bug, but for compatibility reasons some internal function cannot be changed, because it is believed to be used by some extensions (although it is not officially exported), thus blocking the fixing of the bug. This TIP goes beyond that, by not having it fixed to some particular consistent behaviour, but to let the script-developer decide on what is the right behaviour for his needs.
Problem 2 prevents the output of binary-outputting programs from being correctly retrieved.
The exec command already has an interface for options, and currently supports one option -keepnewline and the end-of-options marker --. This means that adding a new option will not adversely affect any existing scripts.
This TIP proposes a new option, -binary arg. arg can be either a single boolean value:
if a boolean true then both input (if a << redirection is present) and return value are passed verbatim between Tcl and the external program.
if a boolean false (which is the default) then behaviour would be like it is now, except for input being \0-safe and system-translation taking place as appropriate (e.g. line-endings).
arg can also be one of the keywords in, out, both (which is equivalent to 1) or an empty string (equivalent to 0) for more readable code. The directions are to be seen from external programs perspective.
If arg is any true boolean value, both or out, then the option -keepnewline is implied.
If some usage of exec does not use << string-redirection, then the in-bit has no visible effect.
For now, no binary flag is defined for stderr. This might be subject of a future TIP or left out due to lack of need.
No implementation exists right now, although it is possible that this will change in near future.
An implementation would have to change these functions: Those functions that are not modifyable due to them being used elsewhere, need to be replaced by an extended version, and the old function can be changed to call the new one with appropriate extra arguments, and eventually be phased out.
Handle the new option and set new bits for the flags argument of Tcl_OpenCommandChannel or add a new bitset argument.
Deal with the new bits, or with new argument and pass them/it on.
needs new arguments flags (for solving bug #768678 along the way) and pass new arguments to a new variant of TclpCreateTempFile.
both Unix and Win version currently get a dumb C-style char-pointer with no length-information. They need two extra arguments, a length and the appropriate binary-bit. Since TclpCreateTempFile is in the stubs-table, we definitely need a new Version of it, e.g. TclpCreateTempFileEx, which will take and use these new arguments.
This document has been placed in the public domain.
This is not necessarily the current version of this TIP.