Category Archives: Open Source

Open Source, Free Software, and similar

libspdy

SPDY is a neat new protocol and possible contender to replace HTTP – at least in some areas and for some use cases. SPDY has been invented and developed mostly by Google engineers.

SPDY allows better usage of fewer TCP connections (since it sends multiple logical streams over a single physical TCP connection) and it helps clients overcome problems with TCP (like how a new connection starts slowly) while at the same time reducing latency and bandwidth requirements. Very similar to how channels are handled over an SSH connection.SPDY

Chrome of course already supports SPDY and Firefox has some early experimental support being worked on.

Of course there are also legitimate criticisms against SPDY as well, including subjects like how it makes caching proxies impossible (because everything goes over SSL), how it makes debugging a lot harder by using compressed headers, how it is impossible to extract just a single header from the stream due to its compression approach and how the compression state buffers make each individual stream use more memory than plain old HTTP (plain TCP) ones.

We can expect SPDY<=>HTTP gateways to appear so that nobody gets locked into either side of these protocols.

SPDY will provide faster transfers. libcurl is currently used for speed reasons in many cases. To me, it makes perfect sense to have libcurl use and try to use SPDY instead of HTTP exactly like how the browsers are starting to do it, so that the libcurl using applications will get their contents transferred faster.

My thinking is that we introduce some new magic option(s) that makes libcurl use SPDY, and for normal easy interface transfers it will remain to use a single connection for each new SPDY transfer, but if you use the multi interface and you enable pipelining you’ll instead make libcurl do multiple transfers over the same single SPDY connection (as long as you speak with the same server and port etc). From an application’s stand-point it shouldn’t make any difference, apart from being faster than otherwise. Just like we want it!

Implementation wise, I would like to use a reliable and efficient third-party library for the actual SPDY implementation. If there doesn’t exist any, we make one and run that one independently. I found libspdy, but I found some concerns about it (no mailing list, looks like one-man project, not C89 compliant, no API docs etc). I mailed the libspdy author, I hoping we’d sort out my doubts and then I’d base my continued work on that library.

After some time Thomas Roth, primary libspdy author, responded and during our subsequent email exchange I’ve gotten a restored faith and belief in this library and its direction. Not only did he fix the C89 compliance pretty quickly, he is also promising rather big changes that are pending to get committed within a week or so.

Comforted by what I’ve learned from Thomas, I’ll wait for his upcoming changes and I’ll join the soon to be created mailing list for the libspdy project and I’ll contribute some ideas and efforts to help shape it into the fine SPDY library we all want. I can only encourage other fellow SPDY library interested persons to do the same!

Updated: Join the SPDY library development

RMS to Stockholm – November 8th

On November 8th 2011, foss-sthlm has the honors of welcoming Richard M Stallman to Stockholm and we invite you all to come and listen to what he has to say. RMS, as he is commonly known, is of course the founding father of both GNU and FSF and he has served his role of non-compromising believer in and the torch-bearer of the Free Software movement ever since he started it.

Date: November 8
Time: 18:00
Where: Aula Magna at Stockholm University

To get to enjoy this talk, and to be able to perhaps ask a question of your own, you must register and book your seat. You do this by going to the foss-sthlm nov2011 web page and reading the instructions.

We have this required booking concept for this only to make sure that we don’t overbook the room. Please make sure that you “return” tickets that you won’t use. Please help us pull this event through in an excellent manner.

This event is made possible thanks to our sponsors South Pole AB and DSV. We arrange this in cooperation with the great FSCONS team.

Update: we switched to a much bigger place!

Haxx, the second year

Last year I posted my report of what I and my fellows did at Haxx after the first year of true and real independence. As I probably mentioned before, we registered our company 1997 but it was just a side project for over a decade.Haxx logo

Now, when we’re slowly approaching two years it is time to look back and what we’ve done during the past twelve months and what we’re doing right now.

We have firmly established ourselves even more as expert developers within embedded systems. We’re over and over again being hired by the teams that themselves are hired by companies to provide services or products. During the last twelve months, we’ve written software and software designs for a huge medical equipment company, a small video equipment manufacturer, a major international telecom, a market-leading embedded systems provider and a global chip manufacturer. We’ve debugged simulation software, designed video streaming servers, done video subtitling magic, poked on Linux kernel code and we’ve done old-school 8051 and 16bit x86 assembly. I’ve also managed to do a Embedded Linux development (in user-space) training course – twice. All this, in just the past year!

Haxx was (and presented) at FSCONS in Gothenburg, we went to (and presented at) FOSDEM in Brussels and we went to the Rockbox devcon in London. We did lots of work within the foss-sthlm community.

Oh, and we’ve revamped our logo and graphical design.

Haxx consists of three full-time employed senior expert embedded systems consultants. We’ve all been in the industry for over twenty years: Daniel Stenberg, Björn Stenberg and Linus Nielsen Feltzing.

We continuously work with partners in the area to reach out to new and existing customers. As we’re very small and rather spend our time on working in our actual assignments we appreciate the help with sales and marketing. If you’re in the Stockholm area and ever end up needing devoted and skilled embedded software hackers, call us!

I’m gonna do my very best to make sure we get another great year! I’ll report back and tell you how it went.

curl meetup at Fosdem 2012

The FOSDEM 2012 dates were recently revealed (4-5 February 2012).

A pint of guinness

I’d be happy to arrange a get-together for libcurl hackers at Fosdem this year. To me, Brussels, Belgium seems mid-europe enough to be able to attract a bunch of us:

  • libcurl application users/authors
  • libcurl binding hackers
  • libcurl contributors
  • … and everyone else who’s doing related activities or who just is interested

Potential subjects to discuss at such a meeting:

  • what’s the most important stuff libcurl still lacks?
  • what’s the least documented/understood parts of libcurl?
  • are there shared problems several/many libcurl bindings have to solve?
  • can we improve how we work/develop libcurl and bindings?
  • what kind of beer is best at a curl meetup?
  • [fill in your own curl related subject]

I would like at least 4-5 people voicing interest for this to be worthwhile for me to actually try to do anything. Please speak up on the libcurl mailing list, tweet me or mail me privately! The more people that are interested, the more planning and stuff we’ll do for it.

curl 7.22.0

Another release of curl and libcurl just happened. 7.22.0 is released.

Apart from the 28 something documented bug fixes, we introduce a range of changes that could be noteworthy:

  • Added CURLOPT_GSSAPI_DELEGATION – remember that we explicitly disabled GSSAPI delegation in our previous release due to a security problem. Now we introduce an option for the application to control exactly how to behave.
  • Added support for NTLM delegation to Samba’s winbind daemon helper ntlm_auth. This lets libcurl use the external helper program to do things like NTLM single-sign on.
  • Display notes from setup file in testcurl.pl – provides a way for test clients to provide more information back to the centralized test summary on the primary server.
  • BSD-style lwIP TCP/IP stack experimental support on Windows – there are still flaws in lwIP on windows that prevents it from working properly
  • OpenSSL: Use SSL_MODE_RELEASE_BUFFERS if available – this is basically a way to ask OpenSSL to use less memory
  • –delegation was added to set CURLOPT_GSSAPI_DELEGATION – simply the new option exported to the command line tool
  • nss: start with no database if the selected database is broken – a slightly modified behavior
  • telnet: allow programatic use on Windows – basically making the windows implementation in sync with how the non-windows version already has worked for quite some time

This release is this great thanks to 25 friendly contributors.

cURL

A libcurl postergirl?

google for libcurl

If you click the image you’ll see a full-resolution screendump for my recent search for “libcurl” on google. Where did that (image of a) girl come from? Judging from where it appears on the results page right next to the information about the cURL project you can’t but assume that she’s somehow related to the project.

That’s of course not true. When moving the mouse over the image I get a tooltip with a funny “hair curling” URL and that’s also where a click on the image takes me.

A mighty weird way of presenting a search result if you ask me!

I like a good firmware bump

So I have this TV that I got for Christmas 2009. As it happens the guys at Philips clearly kept fixing the software and removed bugs after that moment. No surprise there really. I’ve been an embedded software developer for some twenty years by now. I know that software never gets “done” and that what ships in products is only what seems to be “good enough” at some point in time. Sometimes of course not even that good.

So the other day I took a photo of my TV firmware version. It shows how the firmware was made in April 2009. I did it during a discussion with a friend who happens to have the exact same TV as I do, and it then of course turns out he has a different (newer) firmware.

Oh right, I wonder if I can upgrade to a newer one? Once I’ve mastered the maze of the Philips web site I eventually found a download link and PDFs that told me how to. The list of fixes since my version was extensive and I noticed a few flaws mentioned that I have actually experienced!

The TV firmware download was a whopping 43MB. I realize this is because it is a full-fledged Linux system with kernel and God knows what else they’ve crammed in there. I decided to give it a closer check! The result of that was a little disappointing. It is quite clearly encrypted after some basic initial header.

hexdump -C firmware image

The data that starts on offset 0x220 is not x86 instructions and in fact nothing in the beginning of the file looks like x86 code (I just ran a quick “objdump -D –target binary -m i386” on the file). Of course, I don’t know what architecture my TV runs so perhaps even checking for x86 is wrong. I know MIPS is popular in DVDs, settop-boxes and related graphics stuff but…. Nah, I decided it really wasn’t worth the effort so I stopped investigating. I have no real intention of hacking on it anyway.

So I instead proceeded to the actual procedure of upgrading the thing.

Unzip the zip file and put the file in the root dir of a FAT32-formatted usb-stick. The instructions of course didn’t say it needs to be FAT32 but I used that and it worked, and I just smug in the dark to how a manufacturer like this just assumes that we would have FAT32 on our usb-sticks…

But I digress. When I inserted the upgrade USB, the TV switched itself off, was dark for a short while and then turned itself on again and showed the firmware upgrade screen.

The process was very fast, just like 30-40 seconds or something like that and then it was done and asked me to remove the “media” and restart. Of course we know that a usb stick is “media” so I removed it from the TV set.

The instructions were very clear that to “restart” the TV I must only press the ON/OFF button on the remote once and only once. So I was careful to do just that… 😉

Nothing strange happened, but after a brief moment of black screen the regular and familiar interface.

I jumped into the firmware version menu to check it out and yes, it shows an updated version now:

I did a quick check to see if I could detect my previous quirks now, but they may really be gone. They’ve been related to sound through HDMI and some graphical “glitches” when feeding the TV with full HD from a laptop.

So, with this firmware that was shipped many months after I got my TV, I seem to have gotten a better product.

I haven’t yet tested this new version to a significant degree so I don’t know yet if I’ve gotten some new nasty side-effects from it, as sometimes these kinds of firmware upgrades really cause you pain when something that formerly used to work so good suddenly turns out to not work that good any longer.

Rockbox Devcon 2011

Rockbox

Hoards of hackers in similar-looking t-shirts with funny logos having the b in front of the K (see below for some sort of explanation) were seen on the streets of London on Friday June 3rd 2011.

Thanks a lot to  Google UK who hosted our Rockbox developers conference this time in central London.

We had some short-time visitors but we were 16-18 reverse engineering happy persons in a single room most of the weekend, where we hacked away on code, whined on the amount of outstanding patches and bugs and generally made a large amount of bad jokes and Monthy Python references.

The happy core team was caught on a picture:

Rockbox team Devcon 2011

On the Saturday we plowed through a lengthy list of discussion points to really make the most of all of us gathering physically. Among the outcomes from that is that we decided we want to change to git, we think a lot of future of Rockbox lies in the app for Android, we keep the Archos support and more. The Android builds are going to get into the build system ASAP and we’re gonna setup a system where (only) trusted build clients will participate in the building of Android builds that will be distributed to users – this since applications on phones will have a much greater risk of causing harm if some “bad guy” would try to infect our system with stupid things.

Dominik “bluebrother” Riebling brought up the very interesting point that none of us had noticed: we have two different logos being used in the project: one with the K being in front of the b (like the one on the web page) and one with the K being behind the b – which is used in SVG logos and on just about all Rockbox t-shirts made so far! If you zoom in on the tshirts on the group picture you’ll see!

We will also start allowing GPLv3 code into Rockbox in order to be able to use espeak, but all our code will remain GPLv2 or later. I could only find a single USB header file left that comes from the Linux source tree and has a GPLv2 only license.

Even more than this was discussed but I figure the rest of the details will be posted properly on rockbox.org for those seriously interested.

All in all, it was a very enjoyable weekend with a lot of fun and great friends. We stayed at a hotel just a few blocks from the devcon office which was really convenient. even though its wakeup routine was a bit non-standard. Peter “petur” D’Hoye took a lot of pictures as usual.

We also managed to break the Tower of Rockbox record.

Daniel "Bagder" Stenberg Rockbox Devcon 2011

The group picture was taken by a Google person I don’t know the name of who helped us out, and the one of me was taken by Peter D’Hoye.

Rockbox bridge and tower

Keeping to the tradition and subtle arts of Rockbox Towers, but doing it with a twist to celebrate the place we have Rockbox devcon 2011, we decided to make a Rockbox bridge.

We started out by gathering all devices we had in the room that can run Rockbox and distributed them on the construction floor area. As the Android app runs fine on tablets now there’s actually a rather good way to get some solid base into the construction…

Many Rockbox devices

Once all material was known, the construction started with a large amount of eager engineers contributing with good and bad ideas and at times very shaky hands:

constructing a Rockbox bridge

(wods, scorche, gevaerts and paumary)

The result, involving an iRiver beneath the bridge catching the digital flow, became what might be the longest Rockbox construction done so far:

Rockbox bridge

Rockbox bridge closeup

After the bridge, the work started on the real stuff. Building the tallest Rockbox tower ever made. After a couple of accidents and crashes, the tireless team managed to break the previous 104 cm record and the new Rockbox tower record is now officially 117cm:

Rockbox devcon 2011 tower 117cm

(Pictures in this post were all taken by Peter D’Hoye.)