TIP #406 Version 1.1: "C" is for Cookie

This is not necessarily the current version of this TIP.

Abstract

The "http" package needs cookie support, especially to support complex modern web authentication protocols. This TIP defines a pluggable interface and a TclOO class that implements that interface so that Tcl programmers can control just what is shared and stored, and where.

Rationale and Design Constraints

Cookies are a feature of many modern web applications. They're short strings stored in HTTP clients on behalf of servers, and which are then sent back to those servers with further requests to those servers. Often (but not universally) these short strings are IDs that are used as database keys to look up a "session object" in a server-side database that holds relevant information; such information can include whether the user has logged in, what color scheme to use in the stylesheet, etc. Of particular relevance to Tcl programmers is the fact that they are often now associated with login information; this means that it is not practical to leave Tcl code without cookie support.

Currently, Tcl programs that handle cookies have to do so manually, examining the otherwise-unparsed HTTP headers to see if anything relevant has been set. As the cookie protocol [1] is quite complex, it is highly desirable to have a centralized implementation of at least the basic parsing and injection code.

The main down-side of adding cookie handling is that definitely increases the ability of web servers to track clients written in Tcl. In particular, there is a danger of cross-application tracking. It is therefore necessary to ensure that the cookie handling mechanism is off by default, and that the locations used to store the cookies are controllable by the Tcl application.

Proposed Change to 'http' Package

I propose to add a new configuration option to the http package:

-cookiejar commandPrefixList

This option (configurable via http::config), can either be an empty string (the default), or it can be a list of words that is a command prefix. An empty string will disable cookie handling: the current http package behavior in relation to cookies (i.e., ignoring them) will prevail. However, if a non-empty list of words is supplied, they will be used as if they were a command (or rather a prefix to be expanded) in the following ways:

For Each Cookie Provided by an HTTP Server

When a cookie is supplied by an HTTP server, it will be reported to the cookie jar command like this:

{*}$commandPrefixList storeCookie cookieName cookieValue optionDictionary

The cookieName and cookieValue are relatively self-explanatory; they represent the name/value pair to store. The optionDictionary contains the parsed cookie options, being these (any of which may be absent):

persistent

Boolean; whether the HTTP server wishes the cookie to persist for longer than the current "session". This is the only mandatory property.

httponly

Boolean; whether the cookie ought to be only used with HTTP connections. (NB: This is unlikely to be something we enforce.)

expires

Timestamp; when a persistent cookie will cease to be interesting to the HTTP server that issued it.

domain

String; what domains should this cookie be sent to.

path

String; what resource paths within the relevant HTTP servers should a cookie be sent to.

secure

Boolean; whether the cookie should only be sent on "secure" connections to the HTTP server. (This typically means "HTTPS is required", but the specification is less-than-clear in this area.)

The current HTTP connection token will be in the calling stack frame at the point that this is called, with name token. The result of this command will be ignored (so long as it is non-exceptional).

When Making an HTTP Request

Each time an HTTP request is made, the cookie store is consulted (prior to the connection being opened) to find out what cookies should be sent as part of the request, like this:

{*}$commandPrefixList getCookies protocol host port path

The protocol is the name of the protocol scheme that will be used to contact the HTTP server (typically http or https), the host is the server's hostname, the port is the server's port number (e.g., 80 for standard HTTP), and the path is the resource path on that server. The query and fragment parts of the URL are never supplied to the cookie store as part of this request, nor are any user identification credentials.

The result is treated as a list of keys and values (i.e., it is expected to be a list with an even number of items in it, with the first key at index 0 and the first value at index 1) and describes the collection of cookies to send. The http package will manage the formatting of the cookies as part of the request to send.

The current HTTP connection token will be in the calling stack frame at the point that this is called, with name token.

A Cookie Jar Implementation

To go with the above specification, this TIP also describes a TclOO class that will be provided to implement the cookie store side of the protocol. This class will be provided in the cookiejar package.

The name of the class will be ::http::cookiejar, and its instances will be cookie stores that participate in the above protocol. The constructor of the class will take an optional argument that names an SQLite database that will be used to store the cookies; if no name is provided, an in-memory database will be used and all cookies will be treated like pure session cookies.

The result is that it will be possible to enable cookie handling in a Tcl script using this:

 package require http
 package require cookiejar

 http::config -cookiejar [http::cookiejar new ~/mycookies.db]

Implementation Notes

It is worth noting that the current cookiejar package will download a list of "bad" domains (i.e., domains that correspond to super-registries, such as com, 'ac.uk or tk) when a new database is constructed. This extensive list of domains [2] is a security feature that prevents the setting of cookies for large numbers of hosts at once, but it is belived sufficiently long that it is excessive for supplying as part of the package directly.

The degree to which it is necessary to update cookie stores from that list has not yet been studied.

Privacy

Cookies are often associated in the public mind with problems with privacy. There are two principal mechanisms provided here to mitigate these (beyond the proposed domain restrictions, which follow recommended Best Practice):

1. No Tcl interpreter will ever have cookie handling enabled by default. There will always need to be an explicit action taken to turn it on.

2. Applications have to pick the name of their cookie stores when creating and installing them; there is no default.

Copyright

This document has been placed in the public domain.

This is not necessarily the current version of this TIP.