All posts by Daniel Stenberg

Rockbox

The Rockbox future is an app

18 Comments

We rarely discuss what the future of Rockbox looks like, as it rarely actually matters. We work on whatever we think is fun and all things are fine.

Now, I’d like to do something different and actually lift my nose and stare towards the horizon. What’s in the future for Rockbox?

Dedicated mp3 players vanish

I claim that the current world of portable mp3 and music players is dying. Within just a few years, there will only be a few low-end players left being manufactured and most portable music will be done on much more capable (CPU and OS wise) devices such as the current smartphones, be it Android, Maemo or iPhone based ones.

While maintaining Rockbox to work and prosper on the existing targets is not a bad thing, the end of the line as a stand-alone firmware is in sight. I say the only viable future for Rockbox when the players go very advanced, is not to make Rockbox handle networking etc, it is to make sure that it can run and function as an app on these new devices and to take advantage of their existing network stacks etc.

HTC Magic

An app for that?

Rockbox as an app has been a story we’ve told the kids around the campfires for a good while by now and yet we haven’t actually seen it take off in any significant way. I’m now building up my own interest in working on making this happen. In a chat after my Rockbox talk at Fosdem 2010, two other core Rockbox developers (Zagor and gevaerts) seemed to agree to the general view that a Rockbox future involves it running as an app.

Out of the existing systems mentioned above, I’d prefer to start this work focused on Android. It has the widest company backing combined with open source and it’s also the most used open phone OS. I don’t think there’s anything that will prevent us from working on all those platforms as the back-bone should be able to remain the same and portable code we already have and use. Heck, it could then also become more of a regular app for common desktops too.

Challenges

Changing Rockbox to become an application instead of a full operating system and application suite involves a lot of changes. Some of the things we need to solve include:

  • We need to make sure we can use the OS’ native threading without any particular performance loss. The simulator shows that there are might be problems with this right in the current code as it runs dog slow on a Nokia n900.
  • The full suite of plugins and games and so on doesn’t really fit that well when Rockbox is just an app in itself.
  • Different screen sizes. Lots of stuff in Rockbox assume a compile-time fixed screen size, while a good phone app would have to be able to work with whatever size the particular happens to feature. I don’t think we need to care about live resizing (yet).

Rockbox

Update, July 2010: The app part II.

Rockbox

My Rockbox talk at Fosdem 2010

2 Comments

As I’ve mentioned several times before in this blog, I did a talk about Rockbox and reverse engineering at Fosdem 2010 Feburary 6-7 in Brussels, and since there was no “pre-arranged” video recording of the talks in the embedded devroom, Peter D’Hoye stepped up and recorded the whole thing using his Nokia n900 phone.

I decided to not make the slides for this talk available separately, as they were more or less the same as the ones I used for my FSCONS 2009 talk, so you can go watch them instead if this video isn’t enough!

Rockbox-talk-Fosdem2010

To view it, I suggest you use VLC or similar and tell it to stream directly from one of these URLs, the file is a 1.1GB one with 848×480 resolution running for 51 minutes. Annoyingly, none of the usual free online video services allow this long ones.

http://www.qnapclub.be/rockbox/fosdem2010_rockbox.mp4

http://download.rockbox.org/movies/fosdem2010_rockbox.mp4

cURL and libcurl, Security

a big curl forward

We’re proudly presenting a major new release of curl and libcurl and we call it 7.20.0.

The primary reason we decided to bump the minor number this time was that we introduce a range of new protocols, but we also did some other rather big works. This is the biggest update to curl and libcurl that have been made in recent years. Let me mention some of the other noteworthy changes and bugfixes:

We fixed a potential security issue, that would occur if an application requested to download compressed HTTP content and told libcurl to automatically uncompress it (CURLOPT_ENCODING) as then libcurl could wrongly call the write callback (CURLOPT_WRITEFUNCTION) with a larger buffer than what is documented to be the maximum size.

TFTP was finally converted to a “proper” protocol internally. By that I mean that it can now be used with the multi interface in an asynchronous way and it has far less special treatments. It is now “just another protocol” basically and that is a good thing. Also, the BLKSIZE problem with TFTP that has haunted us for a while was fixed so I really think this is the best version ever for TFTP in libcurl.

In several different places in the code older versions of libcurl didn’t properly call the progress callback while waiting for some special event to happen. This made the curl tool’s progress meter less responding but perhaps more importantly it prevented apps that use libcurl to abort the transfer during those phases. The affected periods included the ftp connection phase (including the initial FTP commands and responses), waiting for the TCP connect to complete and resolving host names using c-ares.

The DNS cache was found to have at least two bugs that could make entries linger in the database eternally and in another case too long. For apps that use a lot of connections to a lot of hosts, these problems could result in some serious performance punishments when the DNS cache lookups got slower and slower over time.

Users of the funny ftp server drftpd will appreciate that (lib)curl now support the PRET command, which is needed when getting data off such servers in passive mode. It’s a bit of a hack, but what can we do? We didn’t invent it nor can we help that it’s a popular thing to use! 😉

cURL

Open Source, Rockbox

My Fosdem 2010

4 Comments

Friday

Björn and I left work on the Friday afternoon and took a flight down to Brussels, Belgium. After having checked in to our hotel, we met up with Frank from the Rockbox project and we headed to the Fosdem beer event that took place on a pub quite nearby to the hotel.

The Beer event was crowded. I mean really really crowded. But we still managed to get seated and we got fine Belgium beers and we had a good time. We met a few other Swedes that turned out to be the first in a long series of Swedes that were there. Petur from Rockbox joined up there as well and together we went over a fair share of their beer selection…Atomium

Saturday

For us tech guys, the Saturday morning had no really exciting subjects and weirdly enough the morning had only one track and the massive amount of parallel tracks didn’t start until after lunch. This gave us an opportunity to go sight-seeing, and we visited the city square and the Atomium before we headed into the FOSDEM premises and squeezed our way in to a presentation.

Peter Stuge from the Coreboot project explained to us that we were by far too many people crammed into that little room so if one of the responsible guys would come around a fair lot of us would get thrown out of there. With that heads up given, he started his talk and gave us insights in what coreboot is, what it does and so on. I’ve heard Peter talk about this topic before, but he’s still a good talker and the topic still is techy and interesting enough to listen to.

Embedded software development best practices by Adrien Ampelas turned out to be a bit boring. Basically we got the feeling that Adrien re-used a company slide show or something and told the audience a lot of things I bet the majority of people already knew. Yes we know we must use version control. Yes we know we should send patches upstream. No we don’t Fosdem Entryagree with you that there never exist any reason not to use git.

Sascha Hauer from the Barebox project (the project that was previously known as U-Boot v2) told us about this U-Boot project and what they’re trying to accomplish. It seems like an interesting approach to fix some of the worst mistakes of U-Boot but still leverage on all the things U-Boot did right. It’ll be fun to see if it gets adoption from board makers and companies in general. I guess there’s a lot of investment in U-Boot so lots of things will probably stick with that for a long time ahead…

Flash enable BIOS reverse engineering by Luc Verhaegen gave us an insight in the x86 based reverse engineering they do in the Coreboot project to figure out how to enable flashes and to make them possible to write to when you want to upgrade them to use Coreboot. It was only a quick run-through, but my general feeling was still that compared to Rockbox-style reverse engineering, their tasks actually seem a lot easier! Still interesting, as Luc is a good speaker.

Sunday

Sunday morning started earlier than yesterday. Interesting talks started right away, and we actually were too slow at breakfast so we missed the first part of the interesting Introduction to RTEMS talk by Thomas Doerfler. RTEMS is a fully open source RTOS that’s been around for ages and that has some very good realtime skills and can get shrunk to a rather small size. A slight downside with it is its slightly odd license, as it is a GPLv2+ license with a rather big exception that is made to allow proprietary applications link with it. It makes it incompatible with regular GPLv2 code.

The RepRap project was presented by Adrian Bowyer and I must admit that these 3D-printers are mighty cool and even more fun to see and witness in the real world than they are to see on tiny pictures on web sites.

Back in the embedded room, Roberto Jacinto told us about apt-get for android – with GUI which pretty much described the Aptoide project. It has nothing in common with apt: it doesn’t do dependencies and it doesn’t use its file formats. It has some pretty significant bugs still, and it generally seemed like a rather immature project that I’m not even sure I agree is on the right track. I’d rather actually see the real apt-get for android, with or without GUI.

The Cross build systems: Present & Future workshop could’ve become interesting. A lot of projects (PTXdist, Buildroot, Crosstool-NG, Openembedded, Emdebian etc) spoke about what they are, what they hope to do and how they’d like to collaborate. Unfortunately it took a bit too long time so by the time all had presented their projects the time was pretty much up. The most controversial and slightly off-topic of them all was Andy Green (formerly involved in Openmoko) who talked about how we all should stop cross-compiling and build directly on the target instead(!) and how booting Linux shouldn’t need a boot-loader and that designing PCBs with NAND is stupid(! again). I didn’t hear anyone agreeing with his ideas.

Next up was my talk on Rockbox. I did it in about 40 minutes and I think I covered a bit of what Rockbox is and how we work when we work with new potential targets. It later struck that I should perhaps have had a slide about what the future holds etc, but hey I think it went pretty smooth anyway! Peter recorded my talk on his n900 so hopefully it’ll soon be available online somewhere. After my talk we met a lot of guys wanting to talk Rockbox, ask about particular players and so on and it was mighty fun and interesting.

Greg Kroah-Hartman did the final talk and he is a very good and engaging speaker that really can catch the big audience in Fosdem’s biggest room. Write and Submit your first Linux kernel patch is his “standard talk” but he’s doing it so good and with such elegance that it is a pleasure to watch and learn from. And I’ll admit I wasn’t aware of the get_maintainers.pl script in the kernel tree. A very useful little thing!

Reflections

Some conclusions and general thoughts about the event:

Lack of gaps – there’s a problem when all talks in all rooms are made gapless. It makes people get up and leave 5-10 minutes before the end of each talk so that they will get in time to the next talk that will start on the full hour in another room. It causes pretty much all question-sessions towards the end to fail since the questions (and answers) can’t be heard.

Hard to find people – it is such a huge event and lots of people I have no idea what they look like, so trying to meet friends and people I’ve only emailed with or chatted with on IRC is very hard. Name tags would be really cool. I did have some benefitsHaxx from using my shirt with a big Haxx logo on the back since a fair amount of people recognized it and approached me!

Audio systems – the quality of the different rooms varied a lot (not only sound-wise but the sound was what bothered me). Unfortunately for me, the embedded room was one of the worst ones when it came to audio. It was a big room sure, but the biggest room had an excellent audio system and thus proved size is not what matters. In this case, I think a lot was to blame on the actual microphone we had there.

Phone apps – having phone apps with the entire schedule and a little map for each room etc was a great service. The app also reminded us when a talk you had marked as “favorite” was about to start. It was a bit strange though how the android and n900 versions of the app differed. The n900 version was buggy and slow, but it did offer the schedule in a time-based view while the android version only allowed us to view the schedule based on rooms.

Next year – yes. I think it was great fun and I will really try to attend next year again. Hopefully other friends will too, since meeting friends at the place really doubles the fun! Thank you all for a nice event!

Network, Security

The big protocols

1 Comment

OWASP Sweden once again arranged another interesting meeting, this time with three talks.owasp

The title of the meeting on January 21st here in Stockholm called the protocols “the big ones” (but in Swedish) but I have no idea what kind of measurement they’ve used or what the small ones are or what other “big protocols” there might be! 😉

First we got to hear HÃ¥vard Eidnes tell us about BGP and that protocol seems to suffer from its share of security problems with the protocol itself but perhaps even more with the actual implementations as one of the bigger recent BGP-related incidents that was spoken about was about how internal routes were leaked to the outside from Pakistan in Feb 2008 which made them block the entire world’s access to Youtube. This talk also gave us some insights on the “wild west” of international routing and the lack of control and proper knowledge about who’s allowed to route what to where.

There then was a session by Rickard Bellgrim about DNSSEC and even though I’ve heard talks about this protocol in the past I couldn’t but to again feel that man they have a lot of terminology in that world that makes even a basic description fairly hard to keep up with in some parts of it all. And man do they have a lot of signing and keys and fingerprints and trusts going on… Of course DNSSEC is the answer to lots of existing problems with DNS and DNSSEC certainly opens up a range of new fun. The idea to somehow replace the need for ca-certs by storing keys in DNS is interesting, but even though technically working and sound I fear the browser vendors and the CAs of the SSL world won’t be very fast to turn the wheels to roll in that direction. DNSSEC certainly makes name resolving a lot more complicated, and I wonder if c-ares should ever get into that game… And BTW, DNSSEC of course doesn’t take away the fact that specific implementations may still be vulnerable to security flaws.

The last talk of the evening was about SSL, or rather TLS, held by Fredrik Hesse. He gave us a pretty detailed insight into how the protocol works, and then a fairly detailed overview of the flaws discovered during the last year or so, primarily MD5 and rogue ca certs, the null-prefix cert names and the TLS renegotiation bug. I felt good about already knowing just about everything of what he told us. I can also boast with having corrected the speaker afterward at the pub where we were having our post-talk-beers as he was evidently very OpenSSL focused when he spoke about what SSL libraries can and cannot do.

A great evening. And with good beers too. Thanks to the organizers!

Rockbox

Rockbox talk at Fosdem

I’m scheduled to do a talk about Rockbox at FOSDEM 2010 in the embedded devroom. I’ve got it confirmed, even though the schedule for that room is still not up on the fosdem site.

I must admit the planning for the schedule and the talks of Fosdem confuses me greatly so I’m not entirely sure how everything will work at there – this is going to become my first visit to Fosdem.

My talk will be based on and be similar to the talk I did on this topic at FSCONS 2009.

Update: fosdem info about the talk.

Rockbox

FOSDEM, the Free and Open Source Software Developers' European Meeting

Web

cookie order

I’ve previously mentioned the IETF httpstate working group several times, and here’s some insights into one topic we’re currently discussing:

HTTP Cookie: sort order

The current httpstate draft for the updated cookie spec says that cookies that are sent to a server with the Cookie: header in a HTTP request need to be sorted. They must be sorted primarily on path length and secondary on creation-time.

According to others on the http-state mailinglist, that’s exactly what the major browsers do already and they thus think we should document that as that’s how cookies work.

During these discussions I’ve learned that cookies with the same name that have been specified with different paths, will all be sent in the request and thus they need to be sent in a particular order and in fact even the original Netscape “spec” said so. I agree with this and I’ve also modified libcurl to act accordingly.

I hold the position that only cookies that have identical names need this treatment and that’s what we must specify. Implementers however will most likely find that sorting all cookies at once will be easier.

The secondary sort key, the creation-time, is much more questionable. Why would the server care about that? How can they even rely on that?

Previously in this discussion (back in August 2009) I checked other open source cookie implementations to see how they deal with cookie sorting. I learned that perl’s LWP does sort them based on path length but on nothing else. The following tools did no sorting at all: wget, curl, libsoup, pavuk, lftp and aria2. I have no doubts that we can find more if we would search for more implementations in more languages and environments.

Specifying that sorting is a MUST on path length will still keep LWP and the next curl version compliant. Specifying that they MUST sort on creation-date as well will then make all of these projects non-conforming.

One problem I have in the cookie discussion in general is that the (5?) major browsers pretty much all behave the same and while the 5 major browsers have almost 100% of the web browser market for users, we cannot then automatically assume that they have 100% of the HTTP client or cookie-using client market. We just don’t know how many applications, tools and frameworks exist out there that aren’t actually browsers but still are using cookies.

Of course, I want to say that creation-time sorting is pointless but I have nothing to back that up with. No numbers. The other side of the discussion has a bunch of browsers that sort like this already, but no numbers or evidence if servers rely on this or how many that do.

Can any reader of this find a site out there that depends on the cookie order being sorted on creation-date?

(Reminder: the charter for the HTTPSTATE working group is to document existing widely used practices. It is not to solve problems or to fix problems in the existing cookie protocol. We all know and acknowledge that cookies as they currently work are quite flawed and painful.)

Open Source

2010 conferences

What are the good conferences 2010 that I really shouldn’t miss? I’m talking open source, tech and internet protocols. Where are you going? I’m currently planning like this:

Fosdem 6-7 Feb in Brussels Belgium: I’m going and I’m doing a Rockbox talk.

foss-sthlm 24 Feb in Stockholm Sweden: I’m arranging and I’m doing a curl talk. This isn’t really a “conference” but I wanted to mention it anyway!

IETF 77 in Anaheim USA, March 21-26: While it would’ve been a blast to go there, it really doesn’t sync very well with my work schedule and other lifely matters so I’ll pass this one! Sorry all friends whom I otherwise would’ve met there!

OWASP AppSec Research in Stockholm Sweden, June 21-24: since it is in Stockholm and these guys tend to have interesting stuff I just may go. It depends a little on how the program will end up and if I manage to cough up a talk for it.

IETF 78 in Maastricht Netherlands July 25-30: I want to go there and I think the timing is much better for this IETF meeting than the previous few ones. With a little luck we’ll get both the HTTPBIS and the HTTPSTATE groups to have meetings here, and who knows what other fun there will be?!

Slackathon 2010 in August in Stockholm Sweden? It’s not decided yet but I hope they will go for it and I will try to attend. Slackathon reminders.

FSCONS in Gothenburg Sweden Oct/Nov: Since this is our current major open source conference in Sweden I really want to go and I hope to be able to do a talk too. I don’t think the date is set yet, which is a bit unfortunate since November this year is a bit special to me so there will be some other events going on at that time that risk conflicting with FSCONS.

cURL and libcurl, Security, Web

httpstate cookie domain pains

Back in 2008, I wrote about when Mozilla started to publish their effort the “public suffixes list” and I was a bit skeptic.

Well, the problem with domains in cookies of course didn’t suddenly go away and today when we’re working in the httpstate working group in the IETF with documenting how cookies work, we need to somehow describe how user agents tend to work with these and how they should work with them. It’s really painful.

The problem in short, is that a site that is called ‘www.example.com’ is allowed to set a cookie for ‘example.com’ but not for ‘com’ alone. That’s somewhat easy to understand. It gets more complicated at once when we consider the UK where ‘www.example.co.uk’ is fine but it cannot be allowed to set a cookie for ‘co.uk’.

So how does a user agent know that co.uk is magic?

Firefox (and Chrome I believe) uses the suffixes list mentioned above and IE is claimed to have its own (not published) version and Opera is using a trick where it tries to check if the domain name resolves to an IP address or not. (Although Yngve says Opera will soon do online lookups against the suffix list as well.) But if you want to avoid those tricks, Adam Barth’s description on the http-state mailing list really is creepy.

For me, being a libcurl hacker, I can’t of course but to think about Adam’s words in his last sentence: “If a user agent doesn’t care about security, then it can skip the public suffix check”.

Well. Ehm. We do care about security in the curl project but we still (currently) skip the public suffix check. I know, it is a bit of a contradiction but I guess I’m just too stuck in my opinion that the public suffixes list is a terrible solution but then I can’t figure anything that will work better and offer the same level of “safety”.

I’m thinking perhaps we should give in. Or what should we do? And how should we in the httpstate group document that existing and new user agents should behave to be optimally compliant and secure?

(in case you do take notice of details: yes the mailing list is named http-state while the working group is called httpstate without a dash!)