HTTP: Chunked Encoding

In chunked encoding, the content is broken up into a number of chunks; each of which is prefixed by its size in bytes. A zero size chunk indicates the end of the response message. If a server is using chunked encoding it must set the Transfer-Encoding header to "chunked".

Chunked-encoding is not the same as Content-Encoding header. The Content-Encoding header is an entity-body header. since transfer-encodings are a property of the message, not of the entity-body. ("Entity-body" refers to the body or payload [e.g. a JPG image] of an HTTP request [e.g. POST or PUT request] or response).

 

Q: When is chunked encoding really useful?

A: Chunked encoding is useful when a large amount of data is being returned to the client and the total size of the response may not be known until the request has been fully processed. An example of this is generating an HTML table of results from a database query. If you wanted to use the Content-Length header you would have to buffer the whole result set before calculating the total content size. However, with chunked encoding you could just write the data one row at a time back to the client. At the end, you could write a zero-sized chunk when the end of the SQL query is reached.

This is the HTTP header that is sent by the server:

Transfer-Encoding: chunked

In the HTTP 1.1 specification, chunked is the only encoding method supported by the "Transfer-Encoding" header.

 

source

HTTP: digest auth example

This "HTTP Digest Authentication" example from wikipedia was just too good. I had to post this here - a real collector's item! ;)

Step 3 has the crucial part of the whole process - the inclusion of the server's nonce into the MD5 hash computation, which refutes replay attacks.

1 . Client request (no authentication):


GET /dir/index.html HTTP/1.0
Host: localhost


2. Server response:


HTTP/1.0 401 Unauthorised
Server: HTTPd/0.9
Date: Sun, 10 Apr 2005 20:26:47 GMT
WWW-Authenticate: Digest realm="testrealm@host.com",
qop="auth,auth-int",
nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
opaque="5ccc069c403ebaf9f0171e9517f40e41"
Content-Type: text/html
Content-Length: 311


"http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">







401 Unauthorised.

3. Client request (user name "Mufasa", password "Circle Of Life"):

GET /dir/index.html HTTP/1.0
Host: localhost
Authorization: Digest username="Mufasa",
realm="testrealm@host.com",
nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
uri="/dir/index.html",
qop=auth,
nc=00000001,
cnonce="0a4f113b",
response="6629fae49393a05397450978507c4ef1",
opaque="5ccc069c403ebaf9f0171e9517f40e41"


4. Server response:

HTTP/1.0 200 OK
Server: HTTPd/0.9
Date: Sun, 10 Apr 2005 20:27:03 GMT
Content-Type: text/html
Content-Length: 7984




(source: http://en.wikipedia.org/wiki/Digest_access_authentication)

web security: nonce

Nonce (wrt HTTP digest authentication)
    
A nonce is a parameter that varies with time. A nonce can be a time stamp, a visit counter on a Web page, or a special marker intended to limit or prevent the unauthorized replay or reproduction of a file.

Because a nonce changes with time, it is easy to tell whether or not an attempt at replay or reproduction of a file is legitimate; the current time can be compared with the nonce. If it does not exceed it or if no nonce exists, then the attempt is authorized. Otherwise, the attempt is not authorized.