TIP #400 Version 1.1: Setting the Compression Dictionary

This is not necessarily the current version of this TIP.

Abstract

Sometimes it is necessary to set the compression dictionary so that a sequence of bytes may be compressed more efficiently (and decompressed as well). This TIP exposes that functionality.

Rationale

The SPDY protocol extensions to HTTP require the seeding of the zlib compression dictionary (which greatly improves the performance of compression on small amounts of data, such as HTTP headers). In order to allow a pure Tcl implementation of the SPDY protocol, it is therefore necessary to provide a mechanism whereby the compression dictionary (a byte-array up to 262 bytes long, according to the zlib documentation).

There is to be no mechanism for retrieving the compression dictionary generated by the compression engine; there is no API for doing that.

Proposed Changes: Tcl

The zlib push command will gain an extra option:

-dictionary bytes

This option will provide a compression dictionary to be used, which will be supplied to the zlib compression engine at the correct moment during compression or provided on request of the compression engine on decompression. The bytes argument will be interpreted as a Tcl bytearray; it must be non-empty if given.

In addition, the zlib stream command will gain some complexity. All the subcommands will gain the ability to take an extra -dictionary bytes pair of options (same interpretation as above), the zlib stream gzip variety will also gain the ability to take -header dict (where dict is a Tcl dictionary such as is passed to the -header option to zlib gzip, not a compression dictionary), and the zlib stream gunzip variety will also gain the ability to take -headerVar name (so that a Tcl dictionary describing the contents of the gzip header can be reported). The omission of the last two were an oversight in TIP #234.

Proposed Change: C

At the C level, one additional function will be provided:

void * Tcl_ZlibStreamGetZstreamp(Tcl_ZlibStream zshandle)

This returns the z_streamp associated with a the given Tcl_ZlibStream structure, which can then be used to directly call appropriate zlib functions not directly exposed through Tcl's interface, notably including deflateSetDictionary and inflateSetDictionary. Note that if a function is exposed through a public interface (e.g., deflate and inflate) then it should not be called via this route or inconsistent things may happen. The return type of Tcl_ZlibStreamGetZstreamp is void* so that there is no need for the zlib public types to form part of Tcl's public API.

Copyright

This document has been placed in the public domain.

This is not necessarily the current version of this TIP.