Previously, in my exciting series “improving the HTTP framing checks in Firefox” we learned that I landed a patch, got it backed out, struggled to improve the checks and finally landed the fixed version only to eventually get that one backed out as well.
And now I’ve landed my third version. The amendment I did this time:
When receiving HTTP content that is content-encoded and compressed I learned that when receiving deflate compression there is basically no good way for us to know if the content gets prematurely cut off. They seem to lack the footer too often for it to make any sense in checking for that. gzip streams however end with a footer so they are easier to reliably detect when they are incomplete. (As was discovered before, the Content-Length: is far too often not updated by the server so it is instead wrongly showing the uncompressed size.)
This (deflate vs gzip) knowledge is now used by the patch, meaning that deflate compressed downloads can be cut off without the browser noticing…
Will this version of the fix actually stick? I don’t know. There’s lots of bad voodoo out there in the HTTP world and I’m putting my finger right in the middle of some of it with this change. I’m pretty sure I’ve not written my last blog post on this topic just yet… If it sticks this time, it should show up in Firefox 39.
Considering how badly implemented deflate seems to be in practice, is there much of a reason to continue supporting it alongside normal gzip compression? I’d expect servers to just fallback to uncompressed content if the browser doesn’t do deflate and the server doesn’t do gzip, but then again I’d also expect them to calculate content-length correctly, so maybe that wouldn’t work great.
Alex: that’s a good question but we haven’t discussed that for this and I think I would still prefer to get this done right first anyway and then we can discuss the future of deflate going forward.