I worked on a patch for Firefox bug 237623 to make sure Firefox would use a stricter check for “HTTP 1.1 framing”, checking that Content-Length is correct and that there’s no broken chunked encoding pieces. I was happy to close an over 10 years old bug when the fix landed in June 2014.
The fix landed and has not caused any grief all the way since June through to the actual live release (Nightlies, Aurora, Beta etc). This change finally shipped in Firefox 33 and I had more or less already started to forget about it, and now things went south really fast.
The amount of broken servers ended up too massive for us and we had to backpedal. The largest amount of problems can be split up in these two categories:
- Servers that deliver gzipped content and sends a Content-Length: for the uncompressed data. This seems to be commonly done with old mod_deflate and mod_fastcgi versions on Apache, but we also saw people using IIS reporting this symptom.
- Servers that deliver chunked-encoding but who skip the final zero-size chunk so that the stream actually never really ends.
We recognize that not everyone can have the servers fixed – even if all these servers should still be fixed! We now make these HTTP 1.1 framing problems get detected but only cause a problem if a certain pref variable is set (network.http.enforce-framing.http1), and since that is disabled by default they will be silently ignored much like before. The Internet is a more broken and more sad place than I want to accept at times.
We haven’t fully worked out how to also make the download manager (ie the thing that downloads things directly to disk, without showing it in the browser) happy, which was the original reason for bug 237623…
Although the code may now no longer alert anything about HTTP 1.1 framing problems, it will now at least mark the connection not due for re-use which will be a big boost compared to before since these broken framing cases really hurt persistent connections use. The partial transfer return codes for broken SPDY and HTTP/2 transfers remain though and I hope to be able to remain stricter with these newer protocols.
This partial reversion will land ASAP and get merged into patch releases of Firefox 33 and later.
Finally, to top this off. Here’s a picture of an old HTTP 1.1 frame so that you know what we’re talking about.
Can’t you check both the proper content length and the decompressed content length in some way? That should take care of the gzip ones at least.
It’s a hack of course, but this is the web…
Yes, that’s one of the decent work-arounds we’re discussing as it should fix the case #1 pretty good.
For the second case, I’m leaning towards being fine with missing a chunk but not being OK with a chunk that was only partly delivered.
The discussion will go on and we’ll how it ends…
Shouldn’t we log errors to the console, at least, for incorrect framing? That would give developers a chance to notice it and move vendors towards fixing it.
Dirkjan: there’s a NS_WARNING() warning at that point in nsHttpTransaction.cpp at least…