Category Archives: Open Source

Open Source, Free Software, and similar

cURL and libcurl, Open Source

To fscons and back in 16 hours

I took the X2000 train to Gothenburg from Stockholm at 08:10 so I was at the conference place first at almost 11:30.

SELF

This meant I got to listen in on the end of Jonas Öberg‘s speech on SELF (an FSF Europe and others project on e-learning and a lot of related matters). This wasn’t really my cup of tea, but the other track had a MySQL talk and that isn’t really my thing so I had to just pick one… 🙂 Nothing bad about the subject or Jonas really, just a hint of where my interests are not so much.

Lunch

At lunch I got the opportunity to catch up with Squid-Henrik (Nordström) to talk about recent happenings in our projects. Squid being about to release v3 after several years, and I could report that curl has gotten support for SSH-related protocols during the last year or so… I also exchanged a few words with Peter Stuge who expressed interest in hacking on libssh2 recently, and well I have done some of that!

Qtopia

Trolltech’s GreenphoneKnut Yrvin from Trolltech Norway did an excellent talk on Qtopia in the Telecom business, which had changed name from something involving Greenphone since they have since ditched that project. Anyway, he spoke of the upcoming possible opportunities for free software and open source in consumer electronics, and then particularly in smart phones and then of course mostly related to Qt and Qtopia. He passed around a Greenphone to let us get a feel or it and fiddle a bit with it and yes, it seemed like a nice phone – not a lot bigger or heavier than my current Sony Ericsson thing. It featured a nice “sliding UI” even if I had serious troubles moving around in the system and I couldn’t really figure out the maneuvering concept much! I suspect all it would take is a little more time and perhaps a manual or someone explaining it to me.

OpenMoko

Neo1973In the subsequent talk Ole Tange from the OpenMoko project also handed out “fiddle-versions” of their primary phone, Neo1973 for us in the audience to touch and hold. The major bad with these were however that both devices were dead so we couldn’t see anything on them, just hold them and feel that… eh yeah. they’re a bit on the biggish side and quite a bit bigger than my current phone in all three dimensions! He spoke about the upcoming 2nd version of the phone that is supposed to become available in Q1 2008 and given that it will feature wifi, bluetooth, accelerometers, GPS, 640×480 pixels touch screen, accelerated graphics card and a mini-B USB plug and run an entirely open and free Linux version with documented hardware is indeed thrilling. The Neo1973’s size is not attractive, but its internals are. This is in fact a unit I will seriously consider buying/hacking when/if it becomes available for purchase.

Other details in his OpenMoko talk gave me the impression that the software is not yet very far advanced. Like he first made a comparison to the OLPC system with hw and sw items side-by-side both listing as GTK+ based UIs, but then he also mentioned a thanks to Trolltech for having ported their Greenphone Qtopia system to OpenMoko. On my direct question if that wasn’t a bit contradictive since surely they must be focusing on ONE of these graphics/widgets systems for their main development, he went on to rant about how OpenMoko “is a computer” that can “run anything”. I’m not sure, but it certainly gave me the impression that there just is no main development… Where is OpenMoko at right now really? Anyone knows? I guess I should spend some time on researching that, and also investigate a bit on the “running Rockbox on OpenMoko” front…

curl

cURLWhen the time came for my talk, at 15:00 we first had to mess about a bit since the computer I was supposed to borrow to run my presentation on was suddenly gone (and used for the other track’s talk I later learned) but thanks to other people I soon had a replacement and I got on with it.

I know the topic by heart of course, curl being my primary open source project for ten years and I know every bit of it and its history and so on, but making a fine presentation based on that is an entirely different story. Also, giving it in English adds a layer of, well not complexity perhaps, but it makes it all bit more rough in the edges since even though I know English pretty well and all, my vocabulary isn’t the largest and I don’t always find the right synonyms and the phrasing etc when trying to explain or argue for my sake.

Also, since I don’t quite know my own presentation by heart it isn’t really the best possible performance I can do, but what the heck. I tried to present curl and libcurl, what they are and what they’re good for, why people use it and how the development is done and why YOU should use it now and in the future. The guys at fscons got all talks on video so I hope to be able to see myself on video soon and I’ll try to learn from that for my next talk. And of course those of you who weren’t present at fscons will get your chance to see my pale face and listen to my Swedish-accented stumbling English! 😉 Oh, and I had to rush the presentation a bit towards the end when my 45 minutes ran out a little bit faster than I had anticipated, or was it the questions that popped up? Questions are good, since they make me aware the audience is with me and are interested.

I’m not sure if the topic of curl is somewhat boring, or if it felt too technical or what, but I think I had less than 50% of the audience listening. The other talk going on while I spoke was a lightning talk session with a bunch of people.

Here’s the slides from my talk, in a 31 page 500K pdf: http://daniel.haxx.se/curl-20071208.pdf

LinuxBIOS

Slightly dry in my mouth after this, I recharged myself with a cup of coffee and some cinnamon-rolls and walked it to see the next talk. Or rather series of talks since this was a “lightning talks” session where five guys spoke quickly about various topics. They were about web development with perl, a weird ajax system called gaia that seemed to involve a lot of .NET, a web development system of some sorts named makumba, and a quick mentioning of a 10 gigabit full open source router. For me, the most interesting piece was Peter Stuge’s brief talk about LinuxBIOS, what it is, what it does and so on. That’s really a to-the-metal project and I like getting back to earth and on to real stuff. Much of what he said and explained about difficulties with documentation from hardware vendors etc are just so familiar to me based on Rockbox experiences. To the great enjoyment of the audience, Peter’s live demo of LinuxBIOS booting up failed notoriously and after numerous resets it finally booted up and started playing loud music – when the following speaker already was half-through his router presentation!

Closing

I only got to hear the beginnings of the closing talk held by Georg Greve from FSF Europe as I had to leave after 20 minutes or so to catch my cab that took me back to the train station and I was on my way back to Stockholm again on the 18:42 train…

Did I mention that I got a tshirt? I planned to include a picture of the shirt here, but I took a shot with my mobile phone when I got home and the camera in it is just so extremely crappy in low-light situations (even if I had all the lights in the room turned on) so I can’t torture you by including it. I’ll have to make another attempt later or find a link to someone else who did…

In conclusion: even though I only did a quick visit and didn’t get to see that many talks, I liked what I saw and I had fun. It sounded like the guys doing this are seriously planning on doing it again next year. I hope they’ll do and that I’ll manage to do there again, hopefully to do another talk!

Rockbox

I Solved the AMS-Sansas’ Firmware Checksum Puzzle

On my Sansa v2 web page, I’m collecting firmware binaries for these new targets in order to figure them out and kickstart the Rockbox effort for them. All firmware files have a .bin extension.

It is quite clear (by simple human inspection) that the first 0x400 bytes in each .bin file is a header (padded with 0xff bytes), as on the 0x400 index there is the ARM exception vector and then there’s ARM code following.SanDisk marked chip, an AMS AS3525

In the header there are numerous values, but the 32 bit value at index 4 immediately looked like it could be a checksum of some sorts.

We found two very similar firmwares for the M200 model, one for the European and one for the American in which the “checksum” values only differed by 2 even though there were clearly multiple (although not extensive) differences in the files.

A checksum that differs with so little indicates a simple algorithm. With something more fancy, like CRC32 or similar, a very small change in the files would cause a major change in the checksum value. Two checksum values near each other rather hinted on a simple addition, subtraction, xor or similar.

So I did a hexdump of the two files, cut off the headers and ran a ‘diff -u’ on them. That showed me that the first lines that differed (on index 0x15990) looked like this in the euro version:

00 00 a0 e3 a4 40 9f e5 00 00 c1 e5 04 00 a0 e1
00 fb ff eb 04 00 80 e0 45 10 a0 e3 01 10 40 e5

And like this in the US version.

02 00 a0 e3 a4 40 9f e5 00 00 c1 e5 04 00 a0 e1
00 fb ff eb 04 00 80 e0 41 10 a0 e3 01 10 40 e5

The differences are shown in bold above to make them more obvious. Plus 4, minus 2… Or the other way, minus 4 plus 2. That was almost too good to be true! The fact that these particular differences seemed to be 2 when the values were added just have to mean that the checksum is done with addition (if I was lucky). And if so, this was the only change that mattered to the header so therefore the checksum didn’t take the whole file into account…

SanDisk Sansa ClipI wrote up a small tool that would try out some variations of an “addition algorithm” with 32 bit adds and with 8 bit adds and then I tried with XORs instead to the same effect. Then it struck me that the value in the header at index 0x0c was not changing by a lot between firmwares and it had a number which was an index after the change I mentioned above, but before the subsequent changes…

The program still didn’t spit out the right value when I restricted the algorithm to the size mentioned in the header… until I realized my tool didn’t skip the header when it did the checksum, and when I added a 0x400 bytes skip the values matched! It was as simple as that. Here’s checksum.c.

There are still a few other unidentified fields in the header.

Rockbox

Rockbox on iPod Touch

iPod TouchWith the recently published jailbreak for iPod Touch, combined with the SDL port for iPhone there should be little in the way for running Rockbox on it as an application, pretty much in exctly the same way I mentioned how Rockbox could be made to run on mobile phones.

It seems a suitable place to start this venture is the iphone-dev project page.

While the iPhone and iPod Touch aren’t 100% identical internally, it seems they’re similar enough to make the differences possible to ignore. Also, the fact that what everyone does is build applications that run under the normal Apple-provided OS, there’s no need to know or learn how to poke on the actual hardware so subtle differences in audio chips etc is abstracted away by the operating system even for applications put on the unit this way.

Update nov 2008: With the recent developments on the linuxoniphone blog, it looks like an iPod Touch version of Rockbox is now a lot more likely to be possible. Still, nobody has yet volunteered to start this work and I won’t even say that it is likely that anyone will make an attempt.

Rockbox

Rockbox on a Mobile Phone?

Sony Ericsson w580i has no RockboxMany of the hackers involved in the Rockbox project of course have had the dream to hack our phones as well one day to make them start behaving and working more in the way we want them to. Many times people have brought up the possibilities of (and the inherent problems with) porting Rockbox to a mobile phone. Mobile phones being even more locked down protected devices than your ordinary average mp3 player. These days mobile phones get more room for additional data (music and videos etc) and more music and media player abilities so many times Rockbox on the phone would make sense.

With the introduction of Linux-based phones, the chances have greatly improved. A Rockbox “simulator” can already be built for Linux, and that’s basically all Rockbox app-layer code that does everything Rockbox does but on a native Linux/cygwin instead of on a target device. It’s just about only SDL and a POSIX system needed.

While the Rockbox simulator so far in its 5.5 year’s life has had its primary use as just a simulator for Rockbox developers, it should be fairly easy and straight-forward to adapt it to a life as a stand-alone application. I figure the current work with get UI code adapted for use with touch screens also might help.

Here’s some platforms that might be suitable:

Motorola EZX

Motorola E680iMotorola has a series of phones usually referred to as the EZX phone platform (the Motorola E680, E680i, E680g, 780, A1200, ROKR E6, and ROKR E2 phones) that run Linux on an XScale PXA27x (ARM core) and have a working SDL port. A guy from Indonesia known as ‘blackhawk’ has ported Rockbox to it, but the details are scarce and the patches are infrequent and out-of-date. (Yes that’s a license violation.) I also just hate it when people store info about anything in nothing but a (weird and ad-packed) forum… Update: blackhawk just showed up in the Rockbox forum!

Android

Google’s huge effort is an operating system that is Linux-based (on ARM as well), so there’s basically just a fine SDL for it that’s missing before Rockbox can run on it. My suspicion is that Android is so java-focused and targeted so that SDL won’t be a primary goal for any phone that’ll run Android, and most probably it won’t be as easy to install and mess around with native code anyway.

OpenMokoNeo 1973

OpenMoko is another Linux-based operating system and environment for mobile phones, primarily for the Neo1973. SDL support is still only mentioned on their wish list.

Nokia N8x0

Nokia N810While the N800 series isn’t exactly a phone (although you can do IP-telephony over wifi with it), it is an “internet tablet” from Nokia that runs Linux and I would suspect that if it just becomes successful enough there will eventually be a phone based on that platform. And anyway, Rockbox for the N800 series might make sense even while it isn’t a phone… SDL is ported for this target (maemo).

Volunteers

Of course, all these are just possibilities that remain theoretical until there’s actually interested and willing volunteers that step forward and make it happen…

Rockbox

AMS Replied with the AS3525 Data Sheet

SanDisk Sansa Clip

AMS was very friendly and replied to my data sheet requesting email very rapidly, and now I have the data sheet for the AS3525 in my possession. This is good news for an upcoming porting effort to the SanDisk Sansa v2 series of players, but it doesn’t make it all perfectly easy since we still don’t know lots of stuff in them.

The reply even contained these warming words:

“I see your initiative increasingly successful and I just read a good review on PC Magazine. My compliments, an outstanding job!”

If you have one of these players (e200v2, c200v2, m200 v2 or Clip) and you feel like joining this effort, do jump in on the forum and we’ll get something going! I don’t personally have one of these targets, but I’m pondering on getting one…

cURL and libcurl, Network, Open Source

Tunneling with libcurl

As I wrote a while ago, companies using http proxies make people feel a need to break out of their proxies.

Bryan is a friend who recently found out that his company is switching proxy to a different one and apparently both corkscrew and proxytunnel have problems with this new piece, and since libcurl offers quite a lot of functionality to accomplish almost this, a new project was born: curltunnel.libcurl

One immediate benefit of using libcurl is the support for multiple authentication methods, in fact more than any of the above mentioned tools.

However, it seems our first quick stab at making this tool (currently 278 lines of code), made it work for several common cases but… not for Bryan’s new proxy.

The current theory is that the proxy actually checks for SSL traffic and only lets that through, and thus it prevents the ssh server banner to appear when we try to tunnel through the proxy to a remote ssh server on port 443. If further testing proves this correct, we will of course have to add a SSL layer to the mix.

Electronics, Rockbox

Sansa V2 and View Roundup

There’s been eager activity in the Rockbox forums lately, and I’ve also had help from friendly guys over at anythingbutipod. But now things are much clearer on the Hardware front of many of the recent SanDisk Sansa players:

Sansa ViewSanDisk has released new or updated mp3 players this autumn, named Sansa View, Sansa e200 v2, Sansa c200 v2, Sansa m200 and Sansa Clip. I’ve summed up the situation on the pages those links take you.

All models except the View use the same main SoC chip now, the AS3525. Funnily, the new m200 isn’t called v2 or anything but it is still totally new hardware compared to the older model called… m200!

There’s no docs available for the AS3525 (yet), and the firmware format for it is still not figured out. There’s lots of work to be done, so we really can use all the help you can offer!

Open Source

Scandinavian Free Software Award Nominee

2 Comments

Free Software Foundation EuropeFree Software Foundation Europe has started the Scandinavian Free Software Award that is said to be “for Nordic citizens, projects or organisations that make an outstanding contribution to the Free Software movement.”

Yours truly is one among the twenty giants in the list of nominees. I do feel honored to be included there, and I realize that in that company I really don’t deserve to be picked as the winner.

The award is handed out at the upcoming Free Software Conference Scandinavia event, Dec 7-8 2007.

Books, Open Source

Dreaming in Code

1 Comment

Dreaming in Code coverRecently I’ve read the book Dreaming in Code by Scott Rosenberg, which primarily is a novel about the development of the open source project Chandler, but is full of diversions into related areas and side-stories about software, software projects, developments and more.

The project was initiated by Mitch Kapor who founded the Open Source Application Foundation and who’s perhaps most known for having created Lotus 1-2-3 back in 1982 and being co-founder of the Electronic Frontier Foundation in 1990.

Scott writes a lot about software development and makes numerous references to the classic The Mythical Man-Month (which I unfortunately haven’t read yet but I have it lying around at home just waiting…).

It is quite obvious that the guys never really could decide what exactly Chandler should do, and thus they spent ages arguing on the goals and the design and since they’re funded by an organization that (seemingly) doesn’t care about any profit or anything from this project, there was a whole team who could be lagging their project for ages without any real damage being done anywhere. In fact, I still don’t understand what Chandler does and what useful purpose it tries to fill!

Kapor basically spent 5 Million USD out of his own pocket money in an attempt to repeat his former favorite application called Agenda, but without knowing exactly what it should do and how. Does it sound like a solid foundation for how to make a killer app?

It is an interesting read, and after having been in the business as a consultant for more than a decade, I do recognize a lot in the book from real-life experiences. The book doesn’t exaggerate or present a worst-case scenario, it happens to follow a project that is merely a project amongst others that just… doesn’t get things done right.