Documenting de-facto handling of multipart/form-data form field file uploads

[stanvass]

I don't see this documented in any IETF, W3C or WHATWG spec, and I notice browser bugs filed throughout the last 15 or so years, when a browser chooses to follow the word of https://www.ietf.org/rfc/rfc1867.txt and ignored some de-facto behaviors that contradict the spec. What happens is the browser is released, then it breaks a lot of servers and server apps, and the browser is forced to reimplement or revert to the undocumented behavior.

Namely, this part of RFC 1867 contradicts real-world behavior:

File inputs may also identify the file name. The file name may be described using the 'filename' parameter of the "content-disposition" header. This is not required, but is strongly recommended in any case where the original filename is known. This is useful or necessary in many applications.

Despite it says it's necessary in many applications right after, the "this is not required" part strongly underestimates the issue.

In theory:

• Any form field that doesn't have an explicit Content-Type, or the Content-Type is different than text/plain (the default) should be considered a "file upload".
• The "filename" can be omitted if its empty, empty and missing filename is the same thing.

But in practice...

• Any presence of header Content-Type in a form field triggers the behavior of servers to consider the field a "file upload"
• Virtually all mainstream servers require the presence of the filename attribute. If the filename attribute is not present, many servers incorrectly try to interpret the field as a text field. If the filename attribute is present, then the field is considered a file upload field.
• Virtually all mainstream servers require the filename attribute to be non-empty if the file upload is to be considered non-empty (some even ignore any present content if the filename is left empty). And respectively if the filename attribute is present, but empty (i.e. filename=""), it's treated by servers as a signal that this is a file upload field, where the user hasn't specified a file.

Here's a link to the implementation of file uploads in PHP, which powers a good % of the web right now:
https://github.com/php/php-src/blob/c8aa6f3a9a3d2c114d0c5e0c9fdd0a465dbb54a5/main/rfc1867.c

Notice PHP treats the filename attribute as the absolute truth about whether a field is a file upload field (filename attribute present) and if a file is present in it (depending on whether the attribute is an empty string or non-empty string). Many other servers do that.

I'm sorry I can't provide more concrete references as I researched this few weeks ago and I didn't keep links and now I find it hard to find them back. It didn't occur to me to file an issue here, but now I am. I hope this might trigger a discussion and more proper research on the topic.

Note that his de-facto behavior has implications in all unexpected places. For example one Chrome bug (if I remember right) involved Blob FormData uploads not encoding a filename attribute and this causing servers not to treat them as file uploads. The solution was to assign a bogus name to the Blob on upload, i.e. say just a hash of its contents (say CRC32). I remember seeing another similar issue being resolved by assigning names like "Untitled.jpg" to uploads, so the filename attribute is not left empty.

[domenic]

This is specified pretty clearly in the spec; did you look? See https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#constructing-form-data-set step 7:

the file (consisting of the name, the type, and the body) as the value

then https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#multipart/form-data-encoding-algorithm step 4 goes into detail on the encoding requirements and the fact that every field from the form data set must be included.

[stanvass]

This is specified pretty clearly in the spec; did you look? See https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#constructing-form-data-set step 7:

the file (consisting of the name, the type, and the body) as the value

I'm sorry to say that, but I feel that's pretty unclear. Heck the headers/attributes are not even called by their actual name here.

The issue I filed concerns the "filename" attribute of the "Content-Disposition" header, but saying just "name" can be easily confused with the "name" attribute of the "Content-Disposition" header.

In the very next sentence after the one you quoted, the word "the name" is again used:

If there are no selected files, then append an entry to the form data set with the name as the name, the empty string as the value, and application/octet-stream as the type.

This time under "name" we're supposed to understand the "name" attribute of "Content-Disposition", because if the set of files is empty there's no filename to speak of. But in the previous sentence, under "name" we're supported to understand "filename" attribute?

I can't say this is clear at all. In fact, I'm not even sure the spec says what you say it does (and I realize the irony, because probably you've written some or all of that, but it is pretty vague as it is written right now).

Side from the vague wording, the significance and interpretation of the "filename" attribute is not explained here. I realize WHATWG specifications tend to have a more algorithmic than declarative and semantic nature, but the result here is a step that's entirely unclear as to its meaning, impact, and even interpretation by the servers out there. Necessary knowledge is missing from the spec.

then https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#multipart/form-data-encoding-algorithm step 4 goes into detail on the encoding requirements and the fact that every field from the form data set must be included.

I'm sorry but I don't see which part refers to the rules for the "filename" attribute I listed in my issue. In fact, the word "filename" doesn't occur even once on that page. Can you please quote the relevant part?

I went to check the linked RFC7578, and here's what it says:

For form data that represents the content of a file, a name for the file SHOULD be supplied as well, by using a "filename" parameter of the Content-Disposition header field. The file name isn't mandatory for cases where the file name isn't available or is meaningless or private; this might result, for example, when selection or drag-and-drop is used or when the form data content is streamed directly from a device.

This once again says the "filename" attribute is optional and again directly contradicts the real-world behavior of servers parsing file uploads... So I can't say any of the materials linked here address the issue, TBH.

[stanvass]

I found back one of the Mozilla/Firefox bugs that discuss the real-world significance of the "filename" attribute: https://bugzilla.mozilla.org/show_bug.cgi?id=649150

Notable quote:

Ugh. This needs to be specified in the spec for FormData :(

The commit: https://hg.mozilla.org/mozilla-central/rev/c48c74bc146b

[domenic]

I'm sorry but I don't see which part refers to the rules for the "filename" attribute I listed in my issue. In fact, the word "filename" doesn't occur even once on that page. Can you please quote the relevant part?

The requirements there apply to all fields, so there's no need to call out filename specifically.

[stanvass]

@domenic

Can we please align our terminology, as I'm not sure what you mean:

• "Fields" typically refers to form fields (i.e. text fields, file upload fields, etc.)
• "Attributes" refers to semicolon-separated name=value pairs following the main value of the "Content-Disposition" and "Content-Type" headers (and other headers).

I'm talking about the "filename" attribute of the "Content-Disposition" header.

• The word "filename" isn't mentioned on this page once.
• "File names" is mentioned twice with regards to encoding special characters, and zero information about the attribute having to be present when there are no file assigned to a files upload field.

[annevk]

How exactly multipart/form-data is to be generated is https://www.w3.org/Bugs/Public/show_bug.cgi?id=16909 and is indeed an open issue of sorts.

To fix it we need lots of tests for the various edge cases and someone willing to write the algorithm.

[domenic]

I apologize if I am incorrect, but I think this issue ends up at its core being the same as #3223, which we have a stalled pull request for in #3276. I will close it to consolidate all discussion into at most two places, instead of four.

[annevk]

I think this is a more general form of that issue as it's not just about percent-encoding, but also serializing and parsing in general. Some of those did come up in that issue and its PR so maybe it's okay though.