Category Archives: Open Source

Open Source, Free Software, and similar

Firmware Hackers!

I fell over this job ad at LinkedIn that sounds like a perfect match to some of our existing Rockbox hackers:

Job Description

In this position, the individual will work on the common back-end architecture common for the SanDisk product line. The individual will develop the programming specifications, will explore alternate designs, and program, debug and deliver completed firmware. The individual should be fluent in programming in C language, familiar with assembly and Python scripting language programming and should have experience with hard drive or flash device firmware development. The individual will analyze, design, program, debug, modify software and troubleshoot code for firmware (IC embedded code) applications. Work often involves analog and digital hardware and software operating systems.

So send in your resume. And once you get the job, don’t be shy and let’s get Rockbox on some more SanDisk devices! 😉

How much for a bug?

no bugsWarning: blog post with no clear conclusion!

I offer support deals to companies that want to get help with Open Source programs I’ve contributed to. The deals I’ve made so far have primarily involved libcurl, c-ares or libssh2, but that’s basically because those are projects in which I participate a lot in (and maintain) so people find me easily in relation to those projects.

I wouldn’t mind accepting service and support deals for other projects or software products either, as long as they are products I know and am fairly familiar with already and I am not scared of digging in and fixing things under the hood when that is required.

In fact, I could very well consider to offer to fix bugs in any Open Source software. Like a general: if you have a bug in an open source project that you really want fixed and you can’t do it yourself I might be your man. Of course this would be limited to some certain kinds of projects and programs, but it could still include a wide range of software. A lot more than the ones I happen to be involved in at any particular point in time.

But while “a bug” is a fairly easily defined term to a user who can’t make something work in a given program it can be anything from dead simple to downright impossible for a developer to fix. The fact that users many times cannot determine if a “bug” is hard or easy, if it’s a bug or a feature not working on purpose, makes such a business deal very hard to provide.

How to pay to get a bug fixed?

Fixed price per bug? Presumably only tricky bugs would be considered for this so it would require a fairly high fixed price. But then it’ll also never be used for simple bugs either since the fixed price would scare away such use cases. I don’t think a fixed-price scheme works very well for this.One dollar bill

Then we only have a variable price approach left. A common way for a consultant like me is to charge for my time spent on a project: I set an hourly rate, I fix the issue in N hours. I charge hourly rate * N. For smallish projects, this is less attractive to customers. If we have no previous relationship, there’s a trust issue where the customer might not just blindly accept that I worked 10 hours on a task they think sounds easy so they feel overcharged. Also, there’s the risk that I estimate the job to be 2 hours but end up spending 12. My conclusion is that per-hour pricing doesn’t work for this either.

A variable price approach based on something else than number of hours it took for me to fix the problem is therefore needed.

A bug fix is of course worth whatever someone is willing to pay for it. But we don’t know what they are prepared to pay. On the other end, a bug fix can get done by someone for the price he/she is willing to accept to get the job done. So where is the cross section of those two unknown graphs?

I don’t have the answer here. I’m very interested in feedback and suggestions though. If you would pay for a bug fix, how would you like to get the price set?

Conversing through the Internet with cURL and libcurl

I fell over a really nice and friendly introductionary article on curl and libcurl, written by M. Tim Jones, on IBM’s developerWorks site.

I must confess I greatly enjoyed his image showing the network layers and how curl/libcurl fits into the general picture:

curl layers

While of course arguably there is no ‘socket layer’ (as sockets are a pure API) I still think pictures like this serves a good purpose helping to explain how things interconnect. I personally really suck at visualizing things with images so I’m happy when I see this nice work I can borrow ideas from!

Making better advisories

A while ago yet another security flaw was discovered in curl (actually the tenth flaw in more than eleven years) by Scott Cantor. He reported it privately to us. We worked on the issue and in the end I posted an official project cURL security advisory about it. It wasn’t anything out of the ordinary really. Scott did great and we fixed the problem rather promptly and in coordination with vendor-sec etc.

After a security advisory and the accompanying release, something particular always happens. It’s the same every time I’ve done this and there’s really no surprise: one by one the different Linux distros and similar parties start to ship their security advisories and alerts about the same problem and they offer their upgrade paths for their users to get a corrected version of the package.

But I’ll tell you why I think those advisories tend to make me really sad. It’s not because of the flaws they fix or how fast or slow they are to appear. It’s entirely due to the contents of them or perhaps in many times the lack of contents.

When the first distro-based advisory comes out, they often tend not to use the phrasing used in the original advisory (which we’ve crafted on for weeks and coordinated with vendor-sec) but they instead offer their own interpretation. This isn’t necessarily a bad thing, but when the guys simplify matters they tend to blur out the actual cause and make the real issue hard to understand. Not to mention that when the first guy had done a mistake, most others just repeat that without thinking.

Credit to the doers

The craft of hunting down security problems in software and the art of then creating a fix for that problem is very time consuming and takes a fair amount of skill and patience. Yet some people do this. Some of those even track down problems in open source code bases and tell the projects about the issues to give them a chance to fix them before they’re made public.

Those people are good people that we need.

In the open source world, and in fact in a lot of other places too, the just about only reward we can offer guys who do outstanding work like this is with attribution. Give credit where credit is due. Mention the guy who did the job!

Distro advisories are not good

Very often the subsequent advisories go the lazy route and they borrow their advisory explanation from another distro’s advisory. Still not using the original explanation. They like short and not too detailed explanations. Factual errors seem to not be too important.

Very few distro-advisories give any credit to the original guy who found the error. The only one thing we can offer as payment is then neglected and this is more of an established practice than a mistake. All distros do this. At best they refer to a CVE number for the flaw, but CVE numbers have the great disadvantage that they very rarely reveal any particular details about the flaw until a long time after the advisory is made.

Not only do they often not credit the originator, they also rarely link back to the original advisory or even the advisory the originator sent out (sometimes they’re sent out independently) – so getting the full description from the actual upstream project is harder than it has to be. They do however generally  link to their own site, using their own issue number for the security problem. If things are good, you can find references to the original in that web page they link to. I’ve also seen several distro advisories that simply don’t at all mention what patches they’ve applied or what particular upstream changset they’ve backported.

In this latest advisory from curl, the common repeated mistake was that the certificate flaw concerned the Common Name field (and it implied that it was only about that field) which is wrong, and which is why the original advisory didn’t explicitly mention that field. It also affects the subjectAltName field and that’s at least – if not more – as important to address for this particular flaw. The flaw also only concerned curl built to use OpenSSL, which was a fact that was often not mentioned at all.

What I suggest!

That every vendor and Linux distro that ship security advisories do this:

  1. credit the original problem founder/researcher. This way the glory and fame goes to the person(s) who often did a lot of research and hard work.
  2. link to the original advisory so that everyone who wants to can get the info and details from the upstream project and their ideas of what the problems are and what the best fixes or work-arounds might be
  3. fact-check your error/solution description better and don’t just repeat what someone else wrote unless you know that’s an accurate description
  4. don’t repeat others’ simplifications and errors. The act of duplicating someone else’s description is pretty low in general and it would often only be a signal that you haven’t understood the issue in the first place. If you have a rule to not copy others you won’t risk copying their mistakes.

Going full-time Haxx

I realize noHaxxt a lot of you who read my site or blog are aware of my actual real world day-job situation (nor should you have to care), but I still want to let you guys know that I’m ending my employment at CAG Contactor and my intention is to find my way forward with my own company, Haxx AB, as employee number 1.

Haxx has existed for over ten years already, but we’ve so far only used it for stuff on the side that wasn’t full-time nor competing with our day-jobs. Starting in October, I’ll now instead work only for and with Haxx.

I don’t expect much in my actual day to day business to change much as I intend to continue as a contract developer / consultant / hacker doing embedded, Linux, open source and network development as an expert and senior engineer.

So if you want my help, you can continue to contact me the same way as before, and I can offer my services like before! 😉 The only difference is in my end where I get more freedom and control.

This move on my behalf will affect some of you indirectly: I will move a lot of web and other internet-based services from servers owned and run by Contactor to servers owned by Haxx. So, expect a lot of my sites and contents to get some uptime glitches in the upcoming month in my struggle to get things up on the new place(s).

50 hours offline

Several sites in the haxx.se domain and other stuff related to me and my fellows were completely offline for almost 50 hours between August 24th 19:00 UTC and August 26th 20:30 UTC.

The sites affected included the main web sites for the following projects: curl, c-ares, trio, libssh2 and Rockbox. It also affected mailing lists and CVS repositories etc for some of those.

The reason for the outage has been explained by the ISP (Black Internet) to be because of some kind of sabotage. Their explanation given so far (first in Swedish):

Strax efter kl 20 i måndags drabbades Black Internet och Black Internets kunder av ett mycket allvarligt sabotage. Sabotaget gjordes mot flera av våra core-switchar, våra knutpunkter. Detta resulterade i ett mer eller mindre totalt avbrott för oss och våra kunder. Vi har polisanmält händelsen och har ett bra samarbete med dem.

Translated to English (by me) it becomes:

Soon after 8pm on Monday, Black Internet and its customers were struck by a very serious act of sabotage. The sabotage was made against several of our core switches. This resulted in a more or less total disruption of service for us and our customers. We have reported the incident to the police and we have a good cooperation with them.

Do note that you could keep track of this situation by following me on twitter.

It’s good to be back. Let’s hope it’ll take ages until we go away like that again!

Update: according to my sources, someone erased/cleared Black Internet’s core routers and then they learned that they had no working backups so they had to restore everything by hand.

kernel hacker foodfights

The concept of flame wars and public pie throwing is not new in the open source world, and the open nature of the projects make us – the audience – get to see everything. To read every upset word and get to point back to the mails in retrospect.

I don’t think people in the open source community is any particularly more trigger-happy to start the flame wars than people are outside of the openness, but open it is and then we can see it.

I’ve always disliked the harsh attitude and language that seems to have become popular in some circles, and I believe Linus Torvalds himself is part of that movement as he’s often rude, bad-mouthed and very aggressive in his (leadership) style. I think that easily grows into a hostile and unfriendly atmosphere where little room is left for fun, for jest and for helping out among friends.

So even if that is not the reason for the recent developments, here’s two episodes from August 2009:

A short while ago we got to see well-known kernel hacker Alan Cox step down as tty maintainer after an emotional argument on the lkml. The argument there was basically Linus telling Alan he should’ve admitted his error and acted on it earlier than he did.

Nearby, on the mailing list linux-arm-kernel a long-going argument about the management of the actual mailing list itself again sparkled up a fire. The argument in this case have been a long going discussion whether the mailing list Russell King (the main ARM Linux maintainer) runs should be open to allow non-subscribers to post without them needing moderation or not. It ended today with Russell shutting down his lists.

Right now, it seems the linux-arm-kernel list is being transferred over to infradead.org by David Woodhouse to continue its life there, but I don’t think we’ve seen the end of this yet so things may settle differently. There’s also this patch pending which suggests using the linux-arm list on vger.kernel.org.

(Readers should note that I myself don’t take side in any of these arguments.)

fully respect your rights

This is [name removed] writing at Toshiba Corporation.

We are considering using your program curl (http://curl.haxx.se/) in our products. Before going any further, however, we would like to confirm the following so that we are sure to fully respect your rights.

I am so impressed. Thank you Toshiba for being this upfront and courteous when incorporating an open source product. The license is perfectly free and open for you to use curl for this purpose, but the sheer act of this “making sure” gets my 10 points for great business conduct.

Slacka-fun!

A bunch of the local OpenBSD fans here in Stockholm run this one-day event every year, called Slackathon. I missed it last year, but in 2007 I was there (and I did a little talk about open source management) and this year I was eager to participate again.

This year, the event was scheduled to take place immediately after a bunch of core OpenBSD developers had had their “hackathon f2k9” in Stockholm, so they could now boast with a series of very well known and very knowledgeable OpenBSD kernel hackers. As I am really not more than a distant observer of the OpenBSD project this of course put the lights on a lot of dusty corners I had no previous idea about. I’m not really a stranger to kernels and kernel hacking in general, and I must confess I had a great time and the team who spoke of various very detailed kernel topics are charismatic and put on a great show.

So I learned about the terrors of the VFS layer and hacking it (and how they’re working on making all the involved caches dynamically sized). I learned how to do active-active syncing  of pf-based firewalls (basically using two independent firewalls in front of something), or at least how the guys made it work fairly well. Or how the pf firewall was optimized to double its forwarding performance. And I got to hear a few wise words from Theo de Raadt and learn not only about their 6 month release schedules but also their plans and ideas around solving problems with livelocking and more. Not to mention the talk about managing physical memory, or the work to get OpenBSD ported to sparc64s with hardware-based virtualization support.

Taken all the hardcore kernel talks into account, I think my own talk on libssh2 (just before dinner) felt like a very light snack to chew and possibly a tiny bit out of the general topic… Anyway, I gave a quick overview of the project, how it started, why it was started, what it is and a bit how it works etc.

The slides from my slackathon talk. I expect to re-use a fair bunch of that, with some improvements and additions, in my libssh2 talk at FSCONS later this year.

Looking forward to Slackathon 2010!

Pictures from Slackathon 2009 by Vladimir Bogodist.

Me in front of the projector screen doing libssh2 talk

Snaxx 21

HaxxYes!

It is now time to once again leave your dark and dusty corners of your office or closet, bring yourself up to speed on what currency we’re using in this country and then unite with fellow hackers and technologists in Stockholm City during a fine September evening. The entire Haxx team is delighted to inform that Snaxx-21 is about to happen…

Monday, September 28th 2009

Time: around 18:30

Where: see the snaxx site!

As usual we’re informal, and as our friends you’re of course allowed and encouraged to bring other friends who are similar in spirit and who you think would appreciate an event such as this.

When you’ve decided to show up, please email me and say so.

There might even be free t-shirts involved this time!

Oh, and if you are a Stockholmer and didn’t get this invite by mail already, let me know and I’ll add you to the list of people who get this notice by the old trusty RFC822 way.