Merge pull request #2780 from pmeenan/window

[compression-dictionary] Allow for larger ZStandard compression windows.

draft-ietf-httpbis-compression-dictionary.md

@@ -6,6 +6,7 @@ category: std
 consensus: true
 v: 3
 area: ART
+submissiontype: IETF
 workgroup: HTTP
 keyword:
  - compression dictionary
@@ -247,6 +248,7 @@ input=MATCH, and baseURL=BASEURL.
 
 When there are multiple dictionaries that match a given request URL, the client
 MUST pick a single dictionary using the following rules:
+
 1. For clients that support request destinations, a dictionary that specifies
 and matches a "match-dest" takes precedence over a match that does not use a
 destination.
@@ -293,24 +295,60 @@ For example:
 Content-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:
 ~~~
 
-# Negotiating the compression algorithm
+# Dictionary-Compressed Brotli
 
-When a compression dictionary is available for use for a given request, the
-algorithm to be used is negotiated through the regular mechanism for
-negotiating content encoding in HTTP.
+The "br-d" content encoding identifies a resource that is a
+"Dictionary-Compressed Brotli" stream.
 
-The dictionary to use is negotiated separately and advertised in the
-"Available-Dictionary" request header.
+A "Dictionary-Compressed Brotli" stream is a Brotli {{RFC7932}} stream for a
+response that has been compressed with an external dictionary using a
+compression window not larger than 16 MB.
+
+Clients that announce support for br-d content encoding MUST be able to
+decompress resources that were compressed with a window size of up to 16 MB.
+
+With Brotli compression, the full dictionary is available during compression
+and decompression independent of the compression window, allowing for
+delta-compression of resources larger than the compression window.
+
+# Dictionary-Compressed Zstandard
+
+The "zstd-d" content encoding identifies a resource that is a
+"Dictionary-Compressed Zstandard" stream.
 
-## Compression Algorithms
-This document introduces two new content encoding algorithms:
+A "Dictionary-Compressed Zstandard" stream is a Zstandard {{RFC8878}} stream
+for a response that has been compressed with an external dictionary.
 
-- br-d: Brotli {{RFC7932}} using an external compression dictionary and a compression window of not more than 16 MB.
-- zstd-d: Zstandard {{RFC8878}} using an external compression dictionary and a compression window of not more than 8 MB.
+Clients that announce support for zstd-d content encoding MUST be able to
+decompress resources that were compressed with a window size of at least 8 MB
+or 1.25 times the size of the dictionary, which ever is greater, up to a
+maximum of 128 MB.
+
+The window size used will be encoded in the content (currently, this can be expressed
+in powers of two only) and it MUST be lower than this limit. An implementation MAY
+treat a window size that exceeds the limit as a decoding error.
+
+With Zstandard compression, the full dictionary is available during compression
+and decompression until the size of the input exceeds the compression window.
+Beyond that point the dictionary becomes unavailable. Using a compression
+window that is 1.25 times the size of the dictionary allows for full delta
+compression of resources that have grown by 25% between releases while still
+giving the client control over the memory it will need to allocate for a given
+response.
+
+# Negotiating the content encoding
+
+When a compression dictionary is available for use for a given request, the
+encoding to be used is negotiated through the regular mechanism for
+negotiating content encoding in HTTP through the "Accept-Encoding" request
+header and "Content-Encoding" response header.
+
+The dictionary to use is negotiated separately and advertised in the
+"Available-Dictionary" request header and "Content-Dictionary" response header.
 
 ## Accept-Encoding
 
-The client adds the algorithms that it supports to the "Accept-Encoding"
+The client adds the content encodings that it supports to the "Accept-Encoding"
 request header. e.g.:
 
 ~~~ http-message
@@ -319,7 +357,7 @@ Accept-Encoding: gzip, deflate, br, zstd, br-d, zstd-d
 
 ## Content-Encoding
 
-If a server supports one of the dictionary algorithms advertised by the client
+If a server supports one of the dictionary encodings advertised by the client
 and chooses to compress the content of the response using the dictionary that
 the client has advertised then it sets the "Content-Encoding" response header
 to the appropriate value for the algorithm selected. e.g.:
@@ -340,19 +378,21 @@ Vary: accept-encoding, available-dictionary
 
 ## Content Encoding
 
-IANA is asked to enter the following into the "HTTP Content Coding Registry" registry ({{HTTP}}):
+IANA is asked to enter the following into the "HTTP Content Coding Registry"
+registry ({{HTTP}}):
 
 - Name: br-d
-- Description: A stream of bytes compressed using the Brotli protocol with an external dictionary of not more than 16 MB.
+- Description: "Dictionary-Compressed Brotli" data format.
 - Reference: This document
-- Notes: {{compression-algorithms}}
+- Notes: {{dictionary-compressed-brotli}}
 
-IANA is asked to enter the following into the "HTTP Content Coding Registry" registry ({{HTTP}}):
+IANA is asked to enter the following into the "HTTP Content Coding Registry"
+registry ({{HTTP}}):
 
 - Name: zstd-d
-- Description: A stream of bytes compressed using the Zstandard protocol with an external dictionary of not more than 8 MB.
+- Description: "Dictionary-Compressed Zstandard" data format.
 - Reference: This document
-- Notes: {{compression-algorithms}}
+- Notes: {{dictionary-compressed-zstandard}}
 
 ## Header Field Registration