I talked in the Mozilla devroom at FOSDEM 2015. Here are the slides from it. It was recorded on video and I will post a suitable link to that once it becomes available. The talk was meant to be 20 minutes, I think I did it on 22 or something.
Tag Archives: IETF
http2 explained 1.8
I've been updating my "http2 explained" document every no
w and then since my original release of it back in April 2014. Today I put up version 1.8 which is one of the bigger updates in a while:
The HTTP/2 Last Call within the IETF ended yesterday and the wire format of the protocol has remained fixed for quite some time now so it seemed like a good moment.
I updated some graphs and images to make them look better and be more personal, I added some new short sections in 8.4 and I refreshed the language in several places. Also, now all links mentioned in footnotes and elsewhere should be properly clickable to make following them a more pleasant experience. And page numbers!
As always, do let me know if you find errors, have questions on the content or think I should add something!
My first year at Mozilla
January 13th 2014 I started my fi
rst day at Mozilla. One year ago exactly today.
It still feels like it was just a very short while ago and I keep having this sense of being a beginner at the company, in the source tree and all over.
One year of networking code work that really at least during periods has not progressed as quickly as I would've wished for, and I've had some really hair-tearing problems and challenges that have taken me sweat and tears to get through. But I am getting through and I'm enjoying every (oh well, let's say almost every) moment.
During the year I've had the chance to meetup with my team mates twice (in Paris and in Portland) and I've managed to attend one IETF (in London) and two special HTTP2 design meetings (in London and NYC).
openhub.net counts 47 commits by me in Firefox and that feels like counting high. bugzilla has tracked activity by me in 107 bug reports through the year.
I've barely started. I'll spend the next year as well improving Firefox networking, hopefully with a higher turnout this year. (I don't mean to make this sound as if Firefox networking is just me, I'm just speaking for my particular part of the networking team and effort and I let the others speak for themselves!)
Onwards and upwards!
I’m eight months in on my Mozilla adventure
I started working for Mozilla in January 2014. Here's some reflections from my first time as Mozilla employee.
Working from home
I've worked completely from home during some short periods before in my life so I had an idea what it would be like. So far, it has been even better than I had anticipated. It suits me so well it is almost scary! No commutes. No delays due to traffic. No problems ever with over-crowded trains or buses. No time wasted going to work and home again. And I'm around when my kids get home from school and it's easy to receive deliveries all days. I don't think I ever want to work elsewhere again... 🙂
Another effect of my work place is also that I probably have become somewhat more active on social networks and IRC. If I don't use those means, I may spent whole days without talking to any humans.
Also, I'm the only Mozilla developer in Sweden - although we have a few more employees in Sweden. (Update: apparently this is wrong and there's' also a Mats here!)
The freedom
I have freedom at work. I control and decide a lot of what I do and I get to do a lot of what I want at work. I can work during the hours I want. As long as I deliver, my employer doesn't mind. The freedom isn't just about working hours but I also have a lot of control and saying about what I want to work on and what I think we as a team should work on going further.
The not counting hours
For the last 16 years I've been a consultant where my customers almost always have paid for my time. Paid by the hour I spent working for them. For the last 16 years I've counted every single hour I've worked and made sure to keep detailed logs and tracking of whatever I do so that I can present that to the customer and use that to send invoices. Counting hours has been tightly integrated in my work life for 16 years. No more. I don't count my work time. I start work in the morning, I stop work in the evening. Unless I work longer, and sometimes I start later. And sometimes I work on the weekend or late at night. And I do meetings after regular "office hours" many times. But I don't keep track - because I don't have to and it would serve no purpose!
The big code base
I work with Firefox, in the networking team. Firefox has about 10 million lines C and C++ code alone. Add to that everything else that is other languages, glue logic, build files, tests and lots and lots of JavaScript.
It takes time to get acquainted with such a large and old code base, and lots of the architecture or traces of the original architecture are also designed almost 20 years ago in ways that not many people would still call good or preferable.
Mozilla is using Mercurial as the primary revision control tool, and I started out convinced I should too and really get to learn it. But darn it, it is really too similar to git and yet lots of words are intermixed and used as command but they don't do the same as for git so it turns out really confusing and yeah, I felt I got handicapped a little bit too often. I've switched over to use the git mirror and I'm now a much happier person. A couple of months in, I've not once been forced to switch away from using git. Mostly thanks to fancy scripts and helpers from fellow colleagues who did this jump before me and already paved the road.
C++ and code standards
I'm a C guy (note the absence of "++"). I've primarily developed in C for the whole of my professional developer life - which is approaching 25 years. Firefox is a C++ fortress. I know my way around most C++ stuff but I'm not "at home" with C++ in any way just yet (I never was) so sometimes it takes me a little time and reading up to get all the C++-ishness correct. Templates, casting, different code styles, subtleties that isn't in C and more. I'm slowly adapting but some things and habits are hard to "unlearn"...
The publicness and Bugzilla
I love working full time for an open source project. Everything I do during my work days are public knowledge. We work a lot with Bugzilla where all (well except the security sensitive ones) bugs are open and public. My comments, my reviews, my flaws and my patches can all be reviewed, ridiculed or improved by anyone out there who feels like doing it.
Development speed
There are several hundred developers involved in basically the same project and products. The commit frequency and speed in which changes are being crammed into the source repository is mind boggling. Several hundred commits daily. Many hundred and sometimes up to a thousand new bug reports are filed - daily.
yet slowness of moving some bugs forward
Moving a particular bug forward into actually getting it land and included in pending releases can be a lot of work and it can be tedious. It is a large project with lots of legacy, traditions and people with opinions on how things should be done. Getting something to change from an old behavior can take a whole lot of time and massaging and discussions until they can get through. Don't get me wrong, it is a good thing, it just stands in direct conflict to my previous paragraph about the development speed.
In the public eye
I knew about Mozilla before I started here. I knew Firefox. Just about every person I've ever mentioned those two brands to have known about at least Firefox. This is different to what I'm used to. Of course hardly anyone still fully grasp what I'm actually doing on a day to day basis but I've long given up on even trying to explain that to family and friends. Unless they really insist.
Vitriol and expectations of high standards
I must say that being in the Mozilla camp when changes are made or announced has given me a less favorable view on the human race. Almost anything or any chance is received by a certain amount of users that are very aggressively against the change. All changes really. "If you'll do that I'll be forced to switch to Chrome" is a very common "threat" - as if that would A) work B) be a browser that would care more about such "conservative loonies" (you should consider that my personal term for such people)). I can only assume that the Chrome team also gets a fair share of that sort of threats in the other direction...
Still, it seems a lot of people out there and perhaps especially in the Free Software world seem to hold Mozilla to very high standards. This is both good and bad. This expectation of being very good also comes from people who aren't even Firefox users - we must remain the bright light in a world that goes darker. In my (biased) view that tends to lead to unfair criticisms. The other browsers can do some of those changes without anyone raising an eyebrow but when Mozilla does similar for Firefox, a shitstorm breaks out. Lots of those people criticizing us for doing change NN already use browser Y that has been doing NN for a good while already...
Or maybe I'm just not seeing these things with clear enough eyes.
How does Mozilla make money?
Yeps. This is by far the most common question I've gotten from friends when I mention who I work for. In fact, that's just about the only question I get from a lot of people... (possibly because after that we get into complicated questions such as what exactly do I do there?)
curl and IETF
I'm grateful that Mozilla allows me to spend part of my work time working on curl.
I'm also happy to now work for a company that allows me to attend to IETF/httpbis and related activities much better than ever I've had the opportunity to in the past. Previously I've pretty much had to spend spare time and my own money, which has limited my participation a great deal. The support from Mozilla has allowed me to attend to two meetings so far during the year, in London and in NYC and I suspect there will be more chances in the future.
Future
I only just started. I hope to grab on to more and bigger challenges and tasks as I get warmer and more into everything. I want to make a difference. See you in bugzilla.
Http2 interim meeting NYC
On June 5th, around thirty people sat down around a huge table in a conference room on the 4th floor in the Google offices in New York City, with a heavy rain pouring down outside.
It was time for another IETF http2 interim meeting. The attendees were all participants in the HTTPbis work group and came from a wide variety of companies and countries. The major browser vendors were represented there, and so were operators and big service providers and some proxy people. Most of the people who have been speaking up on the mailing list over the last year or so, unfortunately with a couple of people notably absent. (And before anyone asks, yes we are a group where the majority is old males like me.)
Most people present knew many of the others already, which helped to create a friendly familiar spirit and we quickly got started on the Thursday morning working our way through the rather long lits of issues to deal with. When we had our previous interim meeting in London, I think most of us though we would've been further along today but recent development and discussions on the list had actually brought back a lot of issues we though we were already done with and we now reiterated a whole slew of subjects. We weren't allowed to take photographs indoors so you won't see any pictures of this opportunity from me here.
We did close many issues and I'll just quickly mention some of the noteworthy ones here...
Extensions
We started out with the topic of "extensions". Should we revert the decision from Zurich (where it was decided that we shouldn't allow extensions in http2) or was the current state of the protocol the right one? The arguments for allowing extensions included that we'd keep getting requests for new things to add unless we have a way and that some of the recent stuff we've added really could've been done as extensions instead. An argument against it is that it makes things much simpler and reliable if we just document exactly what the protocol has and is, and removing "optional" behavior from the protocol has been one of the primary mantas along the design process.
The discussion went back and forth for a long time, and after almost three hours we had kind of a draw. Nobody was firmly against "the other" alternative but the two sides also seemed to have roughly the same amount of support. Then it was yet again time for the coin toss to guide us. Martin brought out an Australian coin and ... the next protocol draft will allow extensions. Again. This also forces implementation to have to read and skip all unknown frames it receives compared to the existing situation where no unknown frames can ever occur.
BLOCKED as an extension
A rather given first candidate for an extension was the BLOCKED frame. At the time BLOCKED was added to the protocol it was explicitly added into the spec because we didn't have extensions - and it is now being lifted out into one.
ALTSVC as an extension
What received slightly more resistance was the move to move out the ALTSVC frame as well. It was argued that the frame isn't mandatory to support and therefore easily can be made into an extension.
Simplified padding
Another small change of the wire format since draft-12 was the removal of the high byte for padding to simplify. It reduces the amount you can pad a single frame but you can easily pad more using other means if you really have to, and there were numbers presented that said that 255 bytes were enough with HTTP 1.1 already so probably it will be enough for version 2 as well.
Schedule
There will be a new draft out really soon: draft -13. Martin, our editor of the spec, says he'll be able to ship it in a week. That is intended to be the last draft, intended for implementation and it will then be expected to get deployed rather widely to allow us all in the industry to see how it works and be able to polish details or wordings that may still need it.
We had numerous vendors and HTTP stack implementers in the room and when we discussed schedule for when various products will be able to see daylight. If we all manage to stick to the plans. we may just have plenty of products and services that support http2 by the September/October time frame. If nothing major is found in this latest draft, we're looking at RFC status not too far into 2015.
Meeting summary
I think we're closing in for real now and I have good hopes for the protocol and our progress to a really wide scale deployment across the Internet. The HTTPbis group is an awesome crowd to work with and I had a great time. Our hosts took good care of us and made sure we didn't lack any services or supplies. Extra thanks go to those of you who bought me dinners and to those who took me out to good beer places!
My http2 document
Yeah, it will now become somewhat out of date and my plan is to update it once the next draft ships. I'll also do another http2 presentation already this week so I hope to also post an updated slide set soonish. Stay tuned!
Wireshark
My plan is to cooperate with the other Wireshark hackers and help making sure we have the next draft version supported in Wireshark really soon after its published.
curl and nghttp2
Most of the differences introduced are in the binary format so nghttp2 will need to be updated again - it is the library curl uses for the wire format of http2. The curl parts will need some adjustments, for example for Content-Encoding gzip that no longer is implicit but there should be little to do in the curl code for this draft bump.
Bye bye RFC 2616
In August 2007 the IETF HTTPbis work group started to make an update to the HTTP 1.1 specification RFC 2616 (from June 1999) which already was an update to RFC 2068 from 1996. I wasn't part of the effort back then so I didn't get to hear the back chatter or what exactly the expectations were on delivery time and time schedule, but I'm pretty sure nobody thought it would take almost seven long years for the update to reach publication status.
On June 6 2014 when RFC 7230 - RFC 7235 were released, the single 176 page document has turned into 6 documents with a total size that is now much larger, and there's also a whole slew of additional related documents released at the same time.
2616 is deeply carved into my brain so it'll take some time until I unlearn that, plus the fact that now we need to separate our pointers to one of those separate document instead of just one generic number for the whole thing. Source codes and documents all over now need to be carefully updated to instead refer to the new documents.
And the HTTP/2 work continues to progress at high speed. More about that in a separate blog post soon.
More details on the road from RFC2616 until today can be found in Mark Nottingham's RFC 2616 is dead.
Less plain-text is better. Right?
Every connection and every user on the Internet is being monitored and snooped at to at least some extent every now and then. Everything from the casual firesheep user in your coffee shop, an admin in your ISP, your parents/kids on your wifi network, your employer on the company network, your country's intelligence service in a national network hub or just a random rogue person somewhere in the middle of all this.
My involvement in HTTP make me mostly view and participate in this discussion with this protocol primarily in mind, but the discussion goes well beyond HTTP and the concepts can (and will?) be applied to most Internet protocols in the future. You can follow some of these discussions in the httpbis group, the UTA group, the tcpcrypt list on twitter and elsewhere.
IETF just published RFC 7258 which states:
Pervasive Monitoring Is a Widespread Attack on Privacy
Passive monitoring
Most networking surveillance can be done entirely passively by just running the correct software and listening in on the correct cable. Because most internet traffic is still plain-text and readable by anyone who wants to read it when the bytes come flying by. Like your postman can read your postcards.
Opportunistic?
Recently there's been a fierce discussion going on both inside and outside of IETF and other protocol and standards groups about doing "opportunistic encryption" (OE) and its merits and drawbacks. The term, which in itself is being debated and often is said to be better called "opportunistic keying" (OK) instead, is about having protocols transparently (invisible to the user) upgrade plain-text versions to TLS unauthenticated encrypted versions of the protocols. I'm emphasizing the unauthenticated word there because that's a key to the debate. Recently I've been told that the term "opportunistic security" is the term to use instead...
In the way of real security?
Basically the argument against opportunistic approaches tends to be like this: by opportunistically upgrading plain-text to unauthenticated encrypted communication, sysadmins and users in the world will consider that good enough and they will then not switch to using proper, strong and secure authentication encryption technologies. The less good alternative will hamper the adoption of the secure alternative. Server admins should just as well buy a cert for 10 USD and use proper HTTPS. Also, listeners can still listen in on or man-in-the-middle unauthenticated connections if they capture everything from the start of the connection, including the initial key exchange. Or the passive listener will just change to become an active party and this unauthenticated way doesn't detect that. OE doesn't prevent snooping.
Isn't it better than plain text?
The argument for opportunism here is that there will be nothing to the user that shows that it is "upgrading" to something less bad than plain text. Browsers will not show the padlock, clients will not treat the connection as "secure". It will just silently and transparently make passive monitoring of networks much harder and it will force actors who truly want to snoop on specific traffic to up their game and probably switch to active monitoring for more cases. Something that's much more expensive for the listener. It isn't about the cost of a cert. It is about setting up and keeping the cert up-to-date, about SNI not being widely enough adopted and that we can see only 30% of all sites on the Internet today use HTTPS - for these reasons and others.
HTTP:// over TLS
In the httpbis work group in IETF the outcome of this debate is that there is a way being defined on how to do HTTP as specified with a HTTP:// URL - that we've learned is plain-text - over TLS, as part of the http2 work. Alt-Svc is the way. (The header can also be used to just load balance HTTP etc but I'll ignore that for now)
Mozilla and Firefox is basically the only team that initially stands behind the idea of implementing this in a browser. HTTP:// done over TLS will not be seen nor considered any more secure than ordinary HTTP is and users will not be aware if that happens or not. Only true HTTPS connections will get the padlock, secure cookies and the other goodies true HTTPS sites are known and expected to get and show.
HTTP:// over TLS will just silently send everything through TLS (assuming that it can actually negotiate such a connection), thus making passive monitoring of the network less easy.
Ideally, future http2 capable servers will only require a config entry to be set TRUE to make it possible for clients to do OE on them.
HTTPS is the secure protocol
HTTP:// over TLS is not secure. If you want security and privacy, you should use HTTPS. This said, MITMing HTTPS transfers is still a widespread practice in certain network setups...
TCPcrypt
I find this initiative rather interesting. If implemented, it removes the need for all these application level protocols to do anything about opportunistic approaches and it could instead be handled transparently on TCP level! It still has a long way to go though before we will see anything like this fly in real life.
The future will tell
Is this just a fad that will get no adoption and go away or is it the beginning of something that will change how we do protocols in the future? Time will tell. Many harsh words are being exchanged over this topic in many a debate right now...
(I'm trying to stick to "HTTP:// over TLS" here when referring to doing HTTP OE/OK over TLS. This is partly because RFC2818 that describes how to do HTTPS uses the phrase "HTTP over TLS"...)
http2 explained
I'm hereby offering you all the first version of my document explaining http2, the protocol. It features explanations on the background, basic fundamentals, details on the wire format and something about existing implementations and what's to expect for the future.
The full PDF currently boasts 27 pages at version 1.0, but I plan to keep up with the http2 development going further and I'm also kind of thinking that I will get at least some user feedback, and I'll do subsequent updates to improve and extend the document over time. Of course time will tell how good that will work.
The document is edited in libreoffice and that file is available on github, but ODT is really not a format suitable for patches and merges so I hope we can sort out changes with filing issues and sending emails.
Presentation: what is http2
We had the 14th meetup with foss-sthlm yesterday and I talked about http2 for the almost 100 attendees in the audience. See my slides at slideshare:
Http2 from Daniel Stenberg
Reducing the Public Suffix pain?
Let me introduce you to what I consider one of the worst hacks we have in current and modern internet protocols: the Public Suffix List (PSL). This is a list (maintained by Mozilla) with domains that have some kind administrative setup or arrangement that makes sub-domains independent. For example, you can't be allowed to set cookies for "*.com" because .com is a TLD that has independent domains. But the same thing goes for "*.co.uk" and there's no hint anywhere about this - except for the Public Suffix List. Then, take that simple little example and extrapolate to a domain system that grows with several new TLDs every month and more. The PSL is now several thousands of entries long.
And cookies isn't the only thing this is used for. Another really common and perhaps even more important use case is for wildcard matches in TLS server certificates. You should not be allowed to buy and use a cert for "*.co.uk" but you can for "*.yourcompany.co.uk"...
Not really official but still...
If you read the cookie RFC or the spec for how to do TLS wildcard certificate matching you won't read any line putting it crystal clear that the Suffix List is what you must use and I'm sure different browser solve this slightly differently but in practice and most unfortunately (if you ask me) you must either use the list or make your own to be fully compliant with how the web works 2014.
curl, wget and the PSL
In curl and libcurl, we have so far not taken the PSL into account which is by choice since I've not had any decent way to handle it and there are lots of embedded and other use cases that simply won't be able to cope with that large PSL chunk.
Wget hasn't had any PSL awareness either, but the recent weeks this has been brought up on the wget list and more attention has been given to this. Work has been initiated to do something about it, which has lead to...
libpsl
Tim Rühsen took the baton and started the libpsl project and its associated mailing list, as a foundation for something for Wget to use to get PSL awareness.
I've mostly cheered the effort so far and said that I wouldn't mind building on this to enhance curl in the future if it just gets a suitable (liberal enough) license and it seems to go in that direction. For curl's sake, I would like to get a conditional dependency on this so that people without particular size restrictions can use this, and people on more embedded and special-purpose situations can continue to build without PSL support.
If you're interested in helping out in curl and libcurl in this area, feel most welcome!
dbound
Meanwhile, the IETF has set up a new mailing list called dbound for discussions around PSL and similar issues and it seems very timely!