Category Archives: Work

Work stuff

Mozilla, Network, Open Source, Web

licensed to get shared

As my http2 presentation is about to get its 16,000th viewer over at Slideshare I just have to take a moment and reflect over that fact.

Sixteen thousand viewers. I’ve uploaded slides there before over the years but no other presentation has gotten even close to this amount of attention even though some of them have been collecting views for years by now.

http2 presentation screenshot

I wrote my http2 explained document largely due to the popularity of my presentation and the stream of questions and curiosity that brought to life. Within just a couple of days, that 27 page document had been downloaded more than 2,000 times and by now over 5000 times. This is almost 7MB of PDF which I believe raises the bar for the ordinary casual browser to not download it without having an interest and intention to at least glance through it. Of course I realize a large portion of said downloads are never really read.

Someone suggested to me (possibly in jest) that I should convert these into ebooks and “charge 1 USD a piece to get some profit out of them”. I really won’t and I would have a struggle to do that. It has been said before but in my case it is indeed true: I stand on the shoulders of giants. I’ve just collected information and written down texts that mostly are ideas, suggestions and conclusions others have already made in various other forums, lists or documents. I wouldn’t feel right charging for that nor depriving anyone the rights and freedoms to create derivatives and continue building on what I’ve done. I’m just the curator and janitor here. Besides, I already have an awesome job at an awesome company that allows me to work full time on open source – every day.

The next phase started thanks to the open license. A friendly volunteer named Vladimir Lettiev showed up and translated the entire document into Russian and now suddenly the reach of the text is vastly expanded into a territory where it previously just couldn’t penetrate. With using people’s native languages, information can really trickle down to a much larger audience. Especially in regions that aren’t very Englishified.

cURL and libcurl, Mozilla, Network, Open Source, Security

Reducing the Public Suffix pain?

3 Comments

Let me introduce you to what I consider one of the worst hacks we have in current and modern internet protocols: the Public Suffix List (PSL). This is a list (maintained by Mozilla) with domains that have some kind administrative setup or arrangement that makes sub-domains independent. For example, you can’t be allowed to set cookies for “*.com” because .com is a TLD that has independent domains. But the same thing goes for “*.co.uk” and there’s no hint anywhere about this – except for the Public Suffix List. Then, take that simple little example and extrapolate to a domain system that grows with several new TLDs every month and more. The PSL is now several thousands of entries long.

And cookies isn’t the only thing this is used for. Another really common and perhaps even more important use case is for wildcard matches in TLS server certificates. You should not be allowed to buy and use a cert for “*.co.uk” but you can for “*.yourcompany.co.uk”…

Not really official but still…

If you read the cookie RFC or the spec for how to do TLS wildcard certificate matching you won’t read any line putting it crystal clear that the Suffix List is what you must use and I’m sure different browser solve this slightly differently but in practice and most unfortunately (if you ask me) you must either use the list or make your own to be fully compliant with how the web works 2014.

curl, wget and the PSL

In curl and libcurl, we have so far not taken the PSL into account which is by choice since I’ve not had any decent way to handle it and there are lots of embedded and other use cases that simply won’t be able to cope with that large PSL chunk.

Wget hasn’t had any PSL awareness either, but the recent weeks this has been brought up on the wget list and more attention has been given to this. Work has been initiated to do something about it, which has lead to…

libpsl

Tim Rühsen took the baton and started the libpsl project and its associated mailing list, as a foundation for something for Wget to use to get PSL awareness.

I’ve mostly cheered the effort so far and said that I wouldn’t mind building on this to enhance curl in the future if it just gets a suitable (liberal enough) license and it seems to go in that direction. For curl’s sake, I would like to get a conditional dependency on this so that people without particular size restrictions can use this, and people on more embedded and special-purpose situations can continue to build without PSL support.

If you’re interested in helping out in curl and libcurl in this area, feel most welcome!

dbound

Meanwhile, the IETF has set up a new mailing list called dbound for discussions around PSL and similar issues and it seems very timely!

cURL and libcurl, Mozilla, Network

HTTPbis design team meeting London

I’m writing this just hours after the HTTPbis design team meeting in London 2014 has ended.

Around 30 people attended the meeting i Mozilla’s central London office. The fridge was filled up with drinks, the shelves were full of snacks and goodies. The day could begin. This is the Saturday after the IETF89 week so most people attending had already spent the whole or parts of the week before here in London doing other HTTP and network related work. The HTTPbis sessions at the IETF itself were productive and had already pushed us forward.

We started at 9:30 and we quickly got to work. Mark Nottingham guided us through the day with usual efficiency.

We all basically hang out in a huge room, some in chairs, some in sofas and a bunch of people on the floor or just standing up. We had mikes passed around and the http2 discussions were flowing back and forth depending on the topics and what people felt about them. Some of the issues that were nailed down this time and will end up detailed in the upcoming draft-11 are (strictly speaking, we only discussed the things and formed opinions, as by IETF guidelines we can’t decide things on an offline meeting like this):

  • Priorities of streams will have a dependency graph or direction, making individual  streams less or more important than other
  • A client can send headers without compression and tell the proxy that the header shouldn’t be compressed – used a way to mitigate some of the compression security problems
  • There will be no TLS renegotiation allowed mid-session. Basically a client will have to tear down the connection and negotiate again if suddenly a need to use a client certificate arises.
  • Alt-Svc is the way forward so ALTSVC will appear a new frame in draft-11. This is the way to signal to an application that there is another “route” tIMG_20140308_100453o the same content on the same server. This will allow for what is popularly known as “opportunistic encryption” or at least one sort of that. In short, you can do “plain-text” HTTP over a TLS connection using this…
  • We decided that a server should support gzip contents from clients

There were some other things too handled, but I believe those are the main changes. When the afternoon started to turn long, beers and other beverages were brought out and we did enjoy a relaxing social finale of the day before we split up in smaller groups and headed out in the busy London night to get dinner…

Thanks everyone for a great day. I also appreciated meeting several people in real-life I never met before, only discussed with and read emails from online and of course some old friends I hadn’t seen in a long time!

Oh, there’s also a new rough time frame for http2 going forward. Nearest in time would be the draft-11 at the end of March and another interim in the beginning of June (Boston?).

As a reminder, here’s what was happened for draft-10, and here is http2 draft-10.

Out of all people present today, I believe Mozilla was the company with the largest team (8 attendees) – funnily enough none of us Mozillians there actually work in this office or even in this country.

cURL and libcurl, Haxx, Mozilla, Open Source, Rockbox

My FOSDEM 2014

I’m back home after FOSDEM 2014.Lots of coffee A big THANK YOU from me to the organizers of this fine and totally free happening.

Europe’s (the World’s?) biggest open source conference felt even bigger and more crowded this year. There seemed to be more talks that got full, longer lines for food and a worse parking situation.

Nothing of that caused any major concern for me though. I had a great weekend and I met up with a whole busload of friends from all over. Many of them I only meet at FOSDEM. This year I had some additional bonuses by for example meeting up with long-term committers Steve and Dan from the curl project whom I had never met before IRL. Old buddies from Haxx and Rockbox are kind of default! 🙂

Talk-wise this year was also extra good. I’ve always had a soft spot for the Embedded room but this year there was fierce competition for my attention so I spread my time among many rooms and got to see stuff about: clang the compiler, lots of really cool stuff on GDB, valgrind and helgrind, power efficient software, using the GPU to accelerate libreoffice, car automation and open source, how to run Android on low-memory devices, Firefox on Android and more.

I missed out the kdbus talks since it took place in one of them smaller devrooms even though it was “celebrity warning” all over it with Lennart Poettering. In general there’s sometimes this problem at FOSDEM that devrooms have very varying degrees of popularity on the different talks so the size of the room may be too large or too small depending on the separate topics and speakers. But yeah, I understand it is a very hard problem to improve for the organizers.

As a newbie Firefox developer at Mozilla I find it fun to first hear the Firefox on Android talk for an overview on how things  run on that platform now and then I also got references to Firefox both in the helgrind talk and the low-memory Android talk. In both negative and positive senses.

As always on FOSDEM some talks are not super good and we get unprepared speakers who talk quietly, monotone and uninspired but then there’s the awesome people that in spite of accents and the problem of speaking in English as your non-native language, can deliver inspiring and enticing talks that make me just want to immediately run home and try out new things.

The picture on the right is a small tribute to the drinks we could consume to get our spirits up during a talk we perhaps didn’t find the most interesting…

This year I found the helgrind and the gdb-valgrind talks to be especially good together with Meeks’ talk on using the GPU for libreoffice. We generally found that the wifi setup was better than ever before and worked basically all the time.

Accordingly, there were 8333 unique MAC addresses used on the network through the two days, which we then can use to guesstimate the number of attendees. Quite possibly upwards 6000…

See you at FOSDEM 2015. I think I’ll set myself up to talk about something then. I didn’t do any this year.

cURL and libcurl, Development, Mozilla, Network

My first Mozilla week

1 Comment

Working from home

I get up in the morning, shave, eat breakfast and make sure all family members get off as they should. Most days I walk my son to school (some 800 meters) and then back again. When they’re all gone, the house is quiet and then me and my cup of coffee go upstairs and my work day begins.

Systems and accounts

I have spent time this week to setup accounts and sign up for various lists and services. Created profiles, uploaded pictures, confirmed passwords. I’ve submitted stuff and I’ve signed things. There’s quite a lot of systems in use.

My colleagues

I’ve met a few. The Necko team isn’t very big but the entire company is huge and there are just so many people and names. I haven’t yet had any pressing reason to meet a lot of people nor learn a lot of names. I feel like I’m starting out this really slowly and gradually.

Code base

Firefox is a large chunk of code. It takes some 20 minutes to rebuild on my 3.5GHz quad-core Core-i7 with SSD. I try to pull code and rebuild every morning now so that I can dogfood and live on the edge. I also have a bunch of local patches now, some of them which I want to have stewing in my own browser for a while so that I know they at least don’t have any major negative impact!

Figuring out the threading, XPCOM, the JavaScript stuff and everything is a massive task. I really cannot claim to have done more than just scratched the surface so far, but at least I am scratching and I’ve “etagged” the whole lot and I’ve spent some time reading and reviewing code. Attaching a gdb to a running Firefox and checking out behavior and how it looks has also helped.

Netwerk code size

“Netwerk” is the directory name of the source tree where most of the network code is located. It is actually not so ridiculously large as one could fear. Counting only C++ and header files, it sums up to about 220K lines of code. Of course not everything interesting is in this tree, but still. Not mindbogglingly large.

Video conferencing

I’ll admit I’ve not participated in this sort of large scale video conferences before this. With Vidyo and all the different people and offices signed up at once – it is a quite impressive setup actually. My only annoyance so far is that I didn’t get the sound for Vidyo to work for me in Linux with my headphones. The other end could hear me but I couldn’t hear them! I had to defer to using Vidyo on a windows laptop instead.

Doing the video conferencing on a laptop instead of on my desktop machine has its advantages when I do them during the evenings when the rest of the family is at home since then I can move my machine somewhere and sit down somewhere where they won’t disturb me and I won’t disturb them.

Bugzilla

The bug tracker is really in the center for this project, or at least for how I view it and work with it right now. During my first week I’ve so far filed two bug reports and I’ve submitted a suggested patch for a third bug. One of my bugs (Bug 959100 – ParseChunkRemaining doesn’t detect chunk size overflow) has been reviewed fine and is now hopefully about to be committed.

I’ve requested commit access (#961018) as a “level 1” and I’ve signed the committer’s agreement. Level 1 is entry level and only lets me push to the Try server but still, I fully accept that there’s a process to follow and I’m in no hurry. I’ll get to level 3 soon enough I’m sure.

Mercurial

What can I say. After having used it a bit this week without any particularly fancy operations, I prefer git so much more. Of course I’m also much more used to git, but I find that for a lot of the stuff where both have similar concepts I prefer to git way. Oh well, its just a tool. I’ll get around. Possibly I’ll try out the git mirror soon and see if that provides a more convenient environment for me.

curl

What impact did all this new protocol and network code stuff during my work days have on my curl activities?

I got inspired to fix both the chunked encoding parser and the cookie parser’s handling of max-age in libcurl.

What didn’t happen

I feel behind in the implementing-http2 department. I didn’t get my new work laptop yet.

Next weekDaniel's work place

More of the same, land more patches and figure out more code. Grab more smallish bugs others have filed and work on fixing them as more practice.

Also, there’s a HTTPbis meeting in Zürich on Wednesday to Friday that I won’t go to (I’ll spare you the explanation why) but I’ll try to participate remotely.

Mozilla, Open Source

This is my first day at Mozilla

3 Comments

I’m now a Senior Network Engineer at Mozilla. Today is my first day as an employee. Here’s my summary of how I got here and how the process has been so far, from my perspective.

Mozilla is an American company

OK, that’s not news to anyone and neither is it to me. I’m starting off with this because I’m not American. I’m a Swede, and I live in Sweden. When working and communicating with my fellow people at the mothership I of course get the slight cultural differences that are American versus Swedish. It’s not a problem because I’ve gotten quite used to working with Americans at work over the years but going through an entire recruiting and starting-a-new-job process entirely in English with Americans is new to me.

Interviewing

So when I was opening up to my social network to start looking for new assignments since I decided to end my previous one, I was approached by a friend who works for Mozilla. Of course I was interested to work for Mozilla!

So I had a Skype interview with a recruiter first, and then with two engineers. During evenings of course since Mozilla like many other tech companies are mainly in the west coast of the US, meaning -9 hours from me.

After those initial interviews I had to struggle with myself since I was in this luxury position of having no less than two other very interesting projects being suggested. How do you pick the best one out of three really good alternatives? I actually had to wrestle with all the details and factors involved and I decided that this lead was the best out of the three. If Mozilla wanted me, I’d go for that option.

So I took 4 days off from my current work and flew out to Mountain View California one day in November 2013. It takes some 17 hours to get there, I had a spare day to get somewhat adjusted to the time zone and then I fired off no less than five 45 minute interviews in a row that Wednesday. Then I got a ride back to SFO and I took the plane back home.

I’ve been a consultant for 16 years and I’ve done way over 30 projects during this time. I’ve been interviewed for all of them, and a bunch more that I didn’t end up getting. You can say I’m quite experienced in getting interviewed for work. What I’ve learned is to stay honest, just be me and have an as good time as possible but in the end it really isn’t possible to tell how the other end interprets me and my answers and if I match what they desire.

Getting hired

They didn’t reject me. I got the offer. I said yes. I signed the papers.

So one of the most frequent questions I’ve got when I told friends about my new job:

So you’re not gonna be a consultant anymore?

And no, I’m not. This option only came in the form of an employment or not at all, and I decided I wanted to do this rather than the alternatives. I’ve done consultant based development since 1996 and now I’m not anymore! Of course I still have tight bounds to Haxx and I certainly won’t exclude that I’ll return there one day.

My agreement also allows me to spend some time on curl development. Don’t you worry about that. I feel confident that I will only increase my commitment and efforts there.

Getting anxious

Once I had accepted the job I immediately wanted to start and by golly I got a harder time trying to motivate myself do good the period until that assignment ended.

I’ve had more phone meetings, I’ve checked out code and repeatedly rebuilt Firefox since then, I’ve read Bugzilla entries and perhaps most of all I’ve participated in Mozilla’s “onboarding program” which is a web based system that gives me tasks and asks questions and provides information, all in an effort to give them the information about me that they need and give me the proper info on Mozilla, what it is about, its background and what we’re doing ahead. (See how smoothly I used “we” there?)

Anxiously waiting for The Day. This may sound a bit melodramatic to some, but I do kind of feel like my life is changing a bit now into something it wasn’t before.

Starting a new job working remotely at day one

Today is The Day. I assume not everyone of you reading this have done it, but I start my first day at my new work for my new employer on the upper floor of my house, completely alone. I work from home, remotely and I’m also mostly isolated from a large portion of my colleagues by time zones.

So I grab my coffee and walk up the stairs to my home office. I sit down and I dive in. Source code. Lots of source code.

There’s a steep learning curve here, but I’m not afraid of some hard work and I like a challenge, I like code, I like open source and I like internet protocols. And I have good coffee.

Mozilla. I’m here. I’ve started. Today.

Mozilla dinosaur head logo

cURL and libcurl, Haxx, Network, Open Source, Web

I go Mozilla

4 Comments

Mozilla dinosaur head logo

In January 2014, I start working for Mozilla

I’ve worked in open source projects for some 20 years and I’ve maintained curl and libcurl for over 15 years. I’m an internet protocol geek at heart and Mozilla seems like a perfect place for me to continue to explore this interest of mine and combine it with real open source in its purest form.

I plan to use my experiences from all my years of protocol fiddling and making stuff work on different platforms against random server implementations into the networking team at Mozilla and work on improving Firefox and more.

I’m putting my current embedded Linux focus to the side and I plunge into a worldwide known company with worldwide known brands to do open source within the internet protocols I enjoy so much. I’ll be working out of my home, just outside Stockholm Sweden. Mozilla has no office in my country and I have no immediate plans of moving anywhere (with a family, kids and all established here).

I intend to bring my mindset on protocols and how to do things well into the Mozilla networking stack and world and I hope and expect that I will get inspiration and input from Mozilla and take that back and further improve curl over time. My agreement with Mozilla also gives me a perfect opportunity to increase my commitment to curl and curl development. I want to maintain and possibly increase my involvement in IETF and the httpbis work with http2 and related stuff. With one foot in Firefox and one in curl going forward, I think I may have a somewhat unique position and attitude toward especially HTTP.

I’ve not yet met another Swedish Mozillian but I know I’m not the only one located in Sweden. I guess I now have a reason to look them up and say hello when suitable.

Björn and Linus will continue to drive and run Haxx with me taking a step back into the shadows (Haxx-wise). I’ll still be part of the collective Haxx just as I was for many years before I started working full-time for Haxx in 2009. My email address, my sites etc will remain on haxx.se.

I’m looking forward to 2014!

Haxx, Open Source

Parallel Spaghetti Decode Challenge

1 Comment

At the embedded hacking event in GBG yesterday I organized a small contest for the attendees. I’ve done something similar several times before, so I wanted to make it a bit different this time to spice things up a bit. A straight-forward N questions in a row and then a puzzle to get the final question was too easy. I wanted to create a maze or a play-field that you would need to traverse somehow in order to reach the final goal. But it is hard to create a maze that you don’t immediately spot the way through or that you can somehow “cheat” and find the way in other means rather than to actually answer the questions and do right by using your skills… Then I realized that with just a couple of things added, I could fulfill my goals and still get a fun contest. So, let me start by taking you through the first slide that details the rules:

The contest rules

Ok, so to make the rules be a bit clearer we take a look at a simplified example play field so that we understand what we’re about to play on:

A small example play-field

A short summary:

  1. start on a green box
  2. follow the arrow in the direction that your answer to the question of the box leads you. There’s a compass rose there to help you remember the directions! 🙂
  3. each box you visit has a word associated with it, collect the words along the path
  4. when you reach the red box you’ve read the goal and you’re done
  5. then you re-arrange all the box words you’ve collected and create a final question
  6. answer that questions, the fastest to answer wins!

Everything clear? To help the participants, we had both the playfield and the associated questions printed out on two sheets of paper that we handed out together with a pen. The amount of data is just a bit too much to be able to show on a single screen and it may help to use a pen etc to remember the track you take and which words to remember etc. If you want to repeat the exact same situation, you do the same! I did a special black-and-white version of the playfield to make it more printer-friendly. You may want to fire this up in full resolution to get the best experience:

full-spaghetti-playfield-maze

The question sheet looks like this, but click it for the full PDF:

All questions for the challenge

I posted the answers and everything in a separate post!