Category Archives: Open Source

Open Source, Free Software, and similar

curl icons

I got a question on the #curl IRC channel and I have to admit that the curl and libcurl icons are a bit hard to find on the curl web site, so I thought I’d paste them here too to put the lights on them a bit more.

If you use (lib)curl in your projects or site, please consider making this noticeable by mentioning it and possibly using an existing curl icon to show:

powered by libcurl powered by libcurl powered by libcurl powered by libcurl powered by libcurl

powered by libcurl powered by libcurl powered by libcurl

small curl icon small curl icon with border libcurl icon

cURL libcurl

… as you can see there are plenty to choose from! 😉

And please, if you are capable of drawing logos, bitmaps, icons or the likes and you feel like you can do better than these, please step forward and bring on your work!

Parsing those Dreaded PACs

Since a long time companies all over have found it suitable to use “PAC” (proxy auto-config) scripts to control browsers and tell them what proxy to use for what particular URL and more, and this has been a major nuisance for people who use other (automated) means to go through the proxy than one of them huge gui-based browsers.

For a long time, Ralph Mitchell’s take on this was the only half-decent way to get it working. Half-decent only because it was never packaged and provided in a convenient way to people, it was fully functional.

Now, Manu Garg seems to have stepped forward and introduces pacparser which is a C library for parsing PAC files. Period. It looks (very) similar to Ralph’s approach, in that it incorporates the Mozilla’s javascript engine Spidermonkey, with nothing really special added. So, it’s basically the same approach but this time it is packaged, shipped and documented how it is used!

Announced on the curl-users mailing list today!

The Bourne Nmap

Over at insecure.org we can read about nmap‘s appearance in The Bourne Ultimatum (IMDB) movie and they also show two screenshots, out of which I’ll show you one:

I couldn’t resist trying to resolve the host name in there, only to find that telservice.net is a Korean company/network (which kind of makes it less likely to have the address of the Guardian UK – supposedly the hacking target in the movie) and of course the specific host name in this shot doesn’t resolve and the IP address showing isn’t belonging to telservice.net… Wow, who could’ve guessed that? 😉

And yeah, I’m jealous. I want one of the projects I participate in to appear in movies too!

curl keeps connections alive

Just in the last few days we modified curl to enable the SO_KEEPALIVE option on connections it creates. It basically means that curl will now detect connections that are idle after a certain amount of time, even if that time is something around two hours by default and that’s what most systems will have it set to.cURL

The main problem that caused us to finally enable this (you can still disable this by using –no-keep-alive) is when people do (long-lasting) FTP transfers and they use a NAT, firewall or router that detects and removes what it considers are idle connections. An FTP transfer is using two connections, but the control one where the commands are sent over is completely quiet while the actual data transfer is in progress so when the transfer is done, the control connection has been nuked by the router/NAT. Of course curl survives this as good as possible, but it can’t do proper error-checking etc in this situation.

Funnily, there’s no really good fix for the FTP situation since the two hours SO_KEEPALIVE timeout will many times be too long to help (although most modern systems allows you to change the timeout or a system or application level), but the other “obvious” fix is to send a “NOOP” command on the control channel every once in a while during the transfer. But no, that doesn’t work fine either on most servers since it seems the servers often don’t listen on the control connection during the transfer, so all we’d get is curl sending commands that won’t be replied to until the end of the transfer, and thus it will end up causing problems.

Note: curl sets this option. libcurl still doesn’t, so if you want your app to set the option you can use the same CURLOPT_SOCKOPTFUNCTION callback that curl uses. This requires libcurl 7.16.0 or later.

A thankful Rockbox user

I just wanted to share with you some very friendly and encouraging words I received in a mail from an individual that shall remain unidentified. It is actually rather unusual to get this kind of cheerful words as people who get things and enjoy them rarely get back, but instead we hear more from the ones who get problems or are otherwise unhappy. sansa e200R - looks almost like a e200 v2

“From someone not nearly so gifted and talented, thanks for your efforts to develop Rockbox on the Sansa e200 v2.
Your efforts are deeply appreciated by many, and I’m glad that there is someone as clever and kind as you working on this project in your spare time (which I’m sure is precious to you.)
Work on the project when you can, but don’t deprive your family of your time and presence – they are your most important treasure. I’m just a private individual, and I know this little expression of thanks doesn’t mean much, but here it is, just the same. All the very best wishes to you and your family for Christmas and the coming new year!”

Thanks a lot. (linkified by me)

The Scandinavian Free Software Award went to…

skolelinux logoYou might remember that I mentioned that I was nominated for this award. The final winner of it was the Skolelinux project, as was announced on the fsf-europe press-release list just an hour ago (well the prize was handed out on Friday night but I forgot to ask yesterday so I didn’t know until now…). I guess the fscons site will be updated in due time as well…

As I said before, my own contributions to free software is rather tiny and insignificant compared to giants such as this. I think this project is a well-deserved winner of the award. Congratulations to all involved!

To fscons and back in 16 hours

I took the X2000 train to Gothenburg from Stockholm at 08:10 so I was at the conference place first at almost 11:30.

SELF

This meant I got to listen in on the end of Jonas Öberg‘s speech on SELF (an FSF Europe and others project on e-learning and a lot of related matters). This wasn’t really my cup of tea, but the other track had a MySQL talk and that isn’t really my thing so I had to just pick one… 🙂 Nothing bad about the subject or Jonas really, just a hint of where my interests are not so much.

Lunch

At lunch I got the opportunity to catch up with Squid-Henrik (Nordström) to talk about recent happenings in our projects. Squid being about to release v3 after several years, and I could report that curl has gotten support for SSH-related protocols during the last year or so… I also exchanged a few words with Peter Stuge who expressed interest in hacking on libssh2 recently, and well I have done some of that!

Qtopia

Trolltech’s GreenphoneKnut Yrvin from Trolltech Norway did an excellent talk on Qtopia in the Telecom business, which had changed name from something involving Greenphone since they have since ditched that project. Anyway, he spoke of the upcoming possible opportunities for free software and open source in consumer electronics, and then particularly in smart phones and then of course mostly related to Qt and Qtopia. He passed around a Greenphone to let us get a feel or it and fiddle a bit with it and yes, it seemed like a nice phone – not a lot bigger or heavier than my current Sony Ericsson thing. It featured a nice “sliding UI” even if I had serious troubles moving around in the system and I couldn’t really figure out the maneuvering concept much! I suspect all it would take is a little more time and perhaps a manual or someone explaining it to me.

OpenMoko

Neo1973In the subsequent talk Ole Tange from the OpenMoko project also handed out “fiddle-versions” of their primary phone, Neo1973 for us in the audience to touch and hold. The major bad with these were however that both devices were dead so we couldn’t see anything on them, just hold them and feel that… eh yeah. they’re a bit on the biggish side and quite a bit bigger than my current phone in all three dimensions! He spoke about the upcoming 2nd version of the phone that is supposed to become available in Q1 2008 and given that it will feature wifi, bluetooth, accelerometers, GPS, 640×480 pixels touch screen, accelerated graphics card and a mini-B USB plug and run an entirely open and free Linux version with documented hardware is indeed thrilling. The Neo1973’s size is not attractive, but its internals are. This is in fact a unit I will seriously consider buying/hacking when/if it becomes available for purchase.

Other details in his OpenMoko talk gave me the impression that the software is not yet very far advanced. Like he first made a comparison to the OLPC system with hw and sw items side-by-side both listing as GTK+ based UIs, but then he also mentioned a thanks to Trolltech for having ported their Greenphone Qtopia system to OpenMoko. On my direct question if that wasn’t a bit contradictive since surely they must be focusing on ONE of these graphics/widgets systems for their main development, he went on to rant about how OpenMoko “is a computer” that can “run anything”. I’m not sure, but it certainly gave me the impression that there just is no main development… Where is OpenMoko at right now really? Anyone knows? I guess I should spend some time on researching that, and also investigate a bit on the “running Rockbox on OpenMoko” front…

curl

cURLWhen the time came for my talk, at 15:00 we first had to mess about a bit since the computer I was supposed to borrow to run my presentation on was suddenly gone (and used for the other track’s talk I later learned) but thanks to other people I soon had a replacement and I got on with it.

I know the topic by heart of course, curl being my primary open source project for ten years and I know every bit of it and its history and so on, but making a fine presentation based on that is an entirely different story. Also, giving it in English adds a layer of, well not complexity perhaps, but it makes it all bit more rough in the edges since even though I know English pretty well and all, my vocabulary isn’t the largest and I don’t always find the right synonyms and the phrasing etc when trying to explain or argue for my sake.

Also, since I don’t quite know my own presentation by heart it isn’t really the best possible performance I can do, but what the heck. I tried to present curl and libcurl, what they are and what they’re good for, why people use it and how the development is done and why YOU should use it now and in the future. The guys at fscons got all talks on video so I hope to be able to see myself on video soon and I’ll try to learn from that for my next talk. And of course those of you who weren’t present at fscons will get your chance to see my pale face and listen to my Swedish-accented stumbling English! 😉 Oh, and I had to rush the presentation a bit towards the end when my 45 minutes ran out a little bit faster than I had anticipated, or was it the questions that popped up? Questions are good, since they make me aware the audience is with me and are interested.

I’m not sure if the topic of curl is somewhat boring, or if it felt too technical or what, but I think I had less than 50% of the audience listening. The other talk going on while I spoke was a lightning talk session with a bunch of people.

Here’s the slides from my talk, in a 31 page 500K pdf: http://daniel.haxx.se/curl-20071208.pdf

LinuxBIOS

Slightly dry in my mouth after this, I recharged myself with a cup of coffee and some cinnamon-rolls and walked it to see the next talk. Or rather series of talks since this was a “lightning talks” session where five guys spoke quickly about various topics. They were about web development with perl, a weird ajax system called gaia that seemed to involve a lot of .NET, a web development system of some sorts named makumba, and a quick mentioning of a 10 gigabit full open source router. For me, the most interesting piece was Peter Stuge’s brief talk about LinuxBIOS, what it is, what it does and so on. That’s really a to-the-metal project and I like getting back to earth and on to real stuff. Much of what he said and explained about difficulties with documentation from hardware vendors etc are just so familiar to me based on Rockbox experiences. To the great enjoyment of the audience, Peter’s live demo of LinuxBIOS booting up failed notoriously and after numerous resets it finally booted up and started playing loud music – when the following speaker already was half-through his router presentation!

Closing

I only got to hear the beginnings of the closing talk held by Georg Greve from FSF Europe as I had to leave after 20 minutes or so to catch my cab that took me back to the train station and I was on my way back to Stockholm again on the 18:42 train…

Did I mention that I got a tshirt? I planned to include a picture of the shirt here, but I took a shot with my mobile phone when I got home and the camera in it is just so extremely crappy in low-light situations (even if I had all the lights in the room turned on) so I can’t torture you by including it. I’ll have to make another attempt later or find a link to someone else who did…

In conclusion: even though I only did a quick visit and didn’t get to see that many talks, I liked what I saw and I had fun. It sounded like the guys doing this are seriously planning on doing it again next year. I hope they’ll do and that I’ll manage to do there again, hopefully to do another talk!

I Solved the AMS-Sansas’ Firmware Checksum Puzzle

On my Sansa v2 web page, I’m collecting firmware binaries for these new targets in order to figure them out and kickstart the Rockbox effort for them. All firmware files have a .bin extension.

It is quite clear (by simple human inspection) that the first 0x400 bytes in each .bin file is a header (padded with 0xff bytes), as on the 0x400 index there is the ARM exception vector and then there’s ARM code following.SanDisk marked chip, an AMS AS3525

In the header there are numerous values, but the 32 bit value at index 4 immediately looked like it could be a checksum of some sorts.

We found two very similar firmwares for the M200 model, one for the European and one for the American in which the “checksum” values only differed by 2 even though there were clearly multiple (although not extensive) differences in the files.

A checksum that differs with so little indicates a simple algorithm. With something more fancy, like CRC32 or similar, a very small change in the files would cause a major change in the checksum value. Two checksum values near each other rather hinted on a simple addition, subtraction, xor or similar.

So I did a hexdump of the two files, cut off the headers and ran a ‘diff -u’ on them. That showed me that the first lines that differed (on index 0x15990) looked like this in the euro version:

00 00 a0 e3 a4 40 9f e5 00 00 c1 e5 04 00 a0 e1
00 fb ff eb 04 00 80 e0 45 10 a0 e3 01 10 40 e5

And like this in the US version.

02 00 a0 e3 a4 40 9f e5 00 00 c1 e5 04 00 a0 e1
00 fb ff eb 04 00 80 e0 41 10 a0 e3 01 10 40 e5

The differences are shown in bold above to make them more obvious. Plus 4, minus 2… Or the other way, minus 4 plus 2. That was almost too good to be true! The fact that these particular differences seemed to be 2 when the values were added just have to mean that the checksum is done with addition (if I was lucky). And if so, this was the only change that mattered to the header so therefore the checksum didn’t take the whole file into account…

SanDisk Sansa ClipI wrote up a small tool that would try out some variations of an “addition algorithm” with 32 bit adds and with 8 bit adds and then I tried with XORs instead to the same effect. Then it struck me that the value in the header at index 0x0c was not changing by a lot between firmwares and it had a number which was an index after the change I mentioned above, but before the subsequent changes

The program still didn’t spit out the right value when I restricted the algorithm to the size mentioned in the header… until I realized my tool didn’t skip the header when it did the checksum, and when I added a 0x400 bytes skip the values matched! It was as simple as that. Here’s checksum.c.

There are still a few other unidentified fields in the header.