Saying lib out loud

Not being a native English speaker, I’ve always pronounced libcurl with a ‘lib’ part as if it was part of ‘liberty’, and ‘curl’ with a K sound and ending with ‘earl’. I don’t know of any Swedes who would not pronounce ‘lib’ like that, but when speaking Swedish we’re of course highly influenced by other things so it’s not really relevant.

It wasn’t until I got on FLOSS Weekly that I fully realized that some English speakers would actually pronounce the ‘lib’ part as the first syllable of ‘library’ and that does make sense considering lib is short for library.

But libcurl is just a smaller player here. How do you English speakers pronounce libc? libxml? libgcc?

And yes, this is another one of those really important issues in life. Almost as important as how to close parenthetical expressions with emoticons!


Rockbox USB – A Long Journey

The road to get a fully working and enabled mass-storage USB stack in Rockbox has been long and bumpy.

Bridge-Chip

Back in the early days of mp3 players, they all had bridge-chips which basically meant that the entire USB handling was handled by a dedicated chip that also “took control” of the hard drive (right, most of the interesting players used hard drives initially) so Rockbox didn’t have to do much except hand over control, show something nice on the screen in the mean time and then when the plug was pulled, regain control of the drive again and continue working.

But similar to how early players used dedicated HW-solutions to do music playback and then ditched that for pure CPU-powered solutions, the HW-solutions for USB also died out and all recent models do USB in software.

Research

The PortalPlayer chips being notorious for their complete lack of public docs were more or less considered impossibly hard to get USB working on by us. Until MrH came along 2006 and figured out that the USB parts of the PP5024 chip (used in the sansa e200v1 models) was identical to the USB core used by Freescale’s imx31 chip and there are public docs available for that.

Software Stack

The first step towards a software USB stack for Rockbox was taken in the 2007 gsoc, when Christian “austriancoder” Gmeiner took on the task. He went for a Linux-kernel inspired and quite ambitious approach that at the end of the gsoc project still didn’t quite work. The plan was at that point to primarily write a driver for the Sandisk Sansa e200 series, and secondary for other PortalPlayer based targets (such as ipods etc).

Made Working

Enter Björn “Zagor” Stenberg who had a much more minimalistic approach. Not being pleased with the “big” approach Christian had provided, the existing code was simply cut out and replaced with the new leaner version. Focus was then to get something small and simple to work, and then add fancier layers or features later on instead of providing the architecture for that already from the start. The stack started to partially work for mass-storage.

Not Yet Stable

USB wizard Frank Gevaerts joined the effort and worked fiercely on the thing and not only did he bring USB serial support (that does enable things like logf() support to debug rockbox) he also made the mass storage support almost reliable. The non-existing docs for the hardware made the remaining few flaws very hard to catch and the development of the stack stood still at this point for a great while.

Meanwhile, the USB stack could successfully be used on various new (in the Rockbox sense, these models are no longer new on the mp3 player market) targets like the Toshiba Gigabeat S and the Onda series etc. Parts of the Cowon D2 reverse engineering was greatly enhanced thanks to the USB stack as figuring out the FTL is even more tricky without a fine way to get data over to the host PC. All this proved that the stack itself worked nicely but that there were target-specific inits or setups that were missing for the PortalPlayer targets.

Of course the other new targets were not PortalPlayer based so it took efforts from clueful hackers like Vitja Makarov to implement the hardware driver for the TCC chips and Maurus Cuelenaere who did the driver for the Onda.

Stable?

Now, in 2009 our reverse engineering hero Boris “dreamlayers” Gjenero banged his head against the problem and by doing really nice disassembly he figured out previously not done initiation sequences. These have now been committed to the Rockbox SVN respository, and the early tests of this are very promising showing the mass storage support to be very close to get supported “out of the box” on the PortalPlayer 5022 based targets now. The PP5020 ones still seem unstable, and we have no support on the PP5002 ones.

If things just proceed this fine, we’ll be able to ship Rockbox’s own USB stack by default in the upcoming 3.2 release!

Commit

At Wed, 25 Feb 2009 23:40:15 +0100, the USB code was enabled in the SVN code: “Enable USB mass storage on PP5020, PP5022 and PP5024 targets”

Rockbox

Updates: this post was updated several times to gain accuracy and truths pointed out to me by Dave Chapman, Björn Stenberg, Frank Gevaerts, Jens Arnold and others. Thanks!

Project is Standard now

I fell over a (warning: Swedish!) article on the Swedish idg.se site describing the nominees for the yearly award “Guldmusen” (“the golden mouse”). One of them is this year the highly deserving Adam Dunkel, originator of among other things the very cool lwip (light weight IP) open source project.

However, in both articles on IDG that’s not how lwip is described. It is instead claimed to be “an unofficial world standard”.

world standard” huh? Yes I admit that makes it sound quite a few notches better than “well-used and appreciated open source project”.

But lwip is an TCP/IP stack. A very small one. How can that be a standard, even if you call it an unofficial one?

idg.se continues to be rediculed by me and my friends and the reasons are silly things like this. Let’s hope they continue to amuse us for a long time to come! 🙂

You and the 199 others

How many readers are there of my blog, being no frills hard core tech oriented and all?

The stats are pretty clear: there is roughly 200 visitors per day on the actual daniel.haxx.se/blog site. The main RSS feed is requested 600 times per day, but the blog entries are also mirrored and read on advogato.org and fnoss.org as well.

Since I started blogging on this site I’ve done 383 posts (before this one), which makes roughly two posts every three days on average.

Blame game

I’ve mentioned the fancy distributed build system of Rockbox before, and now it just got even fancier as I’ve introduced a “blame” script that sends a mail to the commit mailing list (that gets mailed with a diff of every single commit) every time one or more builds in the automatic system show up red in the build table. Red builds are for errors (as opposed to yellow for just warnings or green for OK)…

The mail simply mentions that the builds are now red and who did the commit(s) that was included in the failing build. Due to the nature of it, it can be one or more committers and in fact there can be more commits than one from any of them. It’ll help us get alerted about the problems, and hopefully get them fixed again a little faster.

Rockbox

Windows localhost slowness

A client of mine and myself ran a bunch of tests doing FTP and SFTP transfers against localhost to measure how fast our custom solution is compared to a set of existing solutions.

The specific results from this aren’t what caught my eyes, mostly because they’re currently still only used for comparisons and to measure relative improvements, but it was instead the relative speed differences between the tests run on Mac 10.5.5, on Windows XP SP3 and on Linux 2.6.26.

Some of the Windows transfers took a magnitude more time than the others. Ten times longer. Since we could see this across multiple tests each being run multiple times and it was also visible with third party tools, the only conclusion I can draw from this is that Windows for some reason has a much slower localhost.

Does any reader of this have any further knowledge or details to share on this topic? Anyone knows if more recent Windows versions do this any better?

It should be noted that on Windows the ssh server used was running in cygwin, which may account for some of the slowness as cygwin isn’t really known for being blazingly fast…

Update:

Three friends responded to this question:

The first mention that he’d got problems on windows in the past where 127.0.0.1 worked but ‘localhost’ didn’t which might indicate that localhost for some reason would be treated differently.

The second said that it has been mentioned that Windows Vista has significant TCP improvements compared to older versions for which version the TCP/IP stack was rewritten completely.

Pierre (at Microsoft) pointed out that on Vista localhost resolves first to ::1 (ipv6) only, which may explain why some people experience quirks on Vista at least. This test was however done on XP…

Open source personal

I participate in a range of different open source projects. Of course I spend more time on some of them and only a very little time in most of them, but I’m currently listed as member of 18 projects on sourceforge and 16 on ohloh and I can easily figure out a bunch more than aren’t listed on either of those sites.

I’m just the kind of guy who tend to actually get the code and write up a patch for problems, and in fact also in many cases I’ll write an fresh application and publish it openly for the world (not that my typical programs get any particularly large audience but still). I’m not saying everyone has to be like this, I’m just describing me here.

It seems this is a troublesome concept for people to grasp.

I get a large amount of private mail where people talk about “your project” (as in a single one that I am supposed to understand which one they’re referring to) and just about all open source-related interview/questionnaire things I’ve filled in tend to assume My One Single Project. In the first case I can often guess which one they refer to by the phrasing of the mail, and in the second case I tend to answer for the project I’m involved the most in.

So I get this feed of private emails on projects I participate in, but I don’t like private emails about open source projects when people request and expect free support and help. If they want free support, I expect the people to post the questions publicly and open to allow others to reply and read both the question and the subsequent answer online, right there at the time they’re asked but also much later when searching for help on the same subject as then the answers will be around in mailing list archives etc.

These days I have a blanket reply form that I bounce back when I get private support mails and I will admit that most people respect that after having been told about the situation. Every now and then of course I get a violent refusal for sympathy and instead I get to learn I’m an arrogant bastard. This is also related to the fact that:

We (Haxx) run and offer commercial support around curl and libcurl, and for that purpose we have a dedicated support email address. Mail there if you’re willing to pay for support. That’s actually quite clearly spelled out everywhere where that address is displayed, but yet people seem to find that a good place to mail random questions and bug reports. Just today I got a very upset mail response after I mentioned the “paid support” part of the deal there expecting us (me?) to instantly fix bugs regardless since I’ve been told about them per email…

All in all, I’m not really complaining since I’m generally getting along fine with everyone and stuff around this.

Just everyone try to keep things apart: the projects, the people and the companies. They’re sometimes intertwined but sometimes not.

tech, open source and networking