All posts by Daniel Stenberg

cURL and libcurl

curl ten years today

1 Comment

Birthdaycake

On March 20th 1998 curl 4 was released. It was the first curl release ever even if already at version 4 since we kept the version number from the previous projects we did before curl – using other names. We started it all with having the tool named httpget (which was an existing small tool written by Rafael Sagula), soon changed name to urlget to end up with curl – all renames happening due to shifting features and focus.

Like many other projects, this started because of an itch. I wanted to get currency rates off the internet to allow an IRC bot to be able to provide an “exchange service” for users with accurate up-to-date rates. I thought the existing projects I found all did too much or did the wrong thing. That bot and service is now gone since long.

curl has been a truly portable project from day 1, and the first windows build was already urlget 2.1 (pre-curl). autoconf support for the build process was added in October 1998.

Unfortunately I don’t have the original release 4 tarball left anymore, the closest one I have is curl 4.8 (dated August 31 1998). curl 4.8 is about 3400 lines of code. Today we’re totaling in well over 100K source lines, so it has grown over 30 times!

I had no big plans for curl nor did I think very much about the future of the project. I just added the features I and my fellow contributors wanted to have for the moment. That’s actually pretty much how the project has continued to work. We don’t have many long-term plans for what to do with it, we mostly look just inches ahead of our noses and act accordingly.

During the version 6 period (Sep 1999 – Mar 2000) we learned that curl was getting popular, was useful and worked rather well, so the work on providing a libcurl started. We wanted to offer other applications the ability to use curl’s file transfer powers. Version 7.1 was released in August 2000 and thus libcurl was officially born.

curl and libcurl remained being a rather low-key project, I just work on it on my spare time and there are no full-time developers paid to work on this project – apart from some occasional sub-projects now and then that have been sponsored by companies and organizations. (See later on for an example.)

Slowly but surely more and more people started using libcurl and contributed with bug reports and patches. When the project turned 5 years in 2003 I collected all the names of all contributors so far and I reached the number 270. I found the number very high and I was mostly kidding when I said I hoped we would double that amount by the time we celebrate our tenth anniversary. Of course we’ve more than doubled that amount today when we have more than 620 named contributors so far – and continuously adding new ones with every release.

During this journey of a decade, I’ve remained the lead developer and project leader but we’re now some 10 developers with commit access (that also use it) and I try to be open and responsive in order to attract more developers to come aboard, to listen to their advice and ideas and to be sensitive on what our users want from us.

In 2005 I was lucky enough to get a grant from the Swedish IIS organization for the purpose of developing a new event-based API for libcurl to better deal with very large amount of connections, the problem so nicely called c10k.

In the days when our humble project turns 10, I spend about two hours spare time per day on the project and it is my primary hobby, we make 5-6 releases per year, we get about 7000 unique visitors on the web site a normal day, about one million curl packages are downloaded per year – from our servers.

Today, libcurl is feature-rich, portable, very widely used, very fast, well supported and there are no signs of stagnation in release nor development pace. In fact, looking at the source-code growth over the last couple of years we can see a pretty stable and continuous growth:

curl source code growth

Just as I never looked ahead and planned for the future much in the past, I don’t do that now either so I really don’t know and can’t tell what the future will hold for us. We’ll just continue to develop the world’s best client-side file transfer library, to make it even more solid for the foreseeable future, to make it do the things users and developers out there think it should do. Possibly that involves adding support for more protocols, removing some of the less popular ones or simply by enhancing how we support the existing ones.

Join the mailing lists and join us for the next ten years to come!

IT Politics, Technology

International money transfers hurt

I have contacts and business associates located literally all over the globe. Some of them occasionally pay me money. Receiving money from abroad is not easy. How can this still be the case in the year 2008 when soon every living soul have mobile phones and we have global mobile broadband just around the corner?

One dollar billThe citizens of US, often called Americans, are too fond of their silly checks and while they often can be squeezed to wire money they tend to not do it. I believe SWIFT/IBAN is a European invention so I suspect the US banks resist only because of that – and of course because the use of checks seems so deeply rooted in the American soil. Cashing in a 4000USD check cost me well over 200USD not very many moons ago. And then I had to go through many hoops at the bank, standing in lines, filling out forms, waiting for clerks not understanding at all what to do but explaining to me that they don’t have to cash it at all so I should better be grateful they do and yes please sign here that you’ll be forced to pay back in case we’ll find out in a hundred years that the check bounces. In Sweden, checks died out in the 80s and I’ve never owned a checkbook and have never used as payment.

From Russia I hear the SWIFT/IBAN thing is also expensive and one of my contacts then prefer to use Western Union instead. Now that’s another sorry and messy approach. I then have to show up at an “agent’s” place and stand in lines, fill in forms, stand waiting for the personal to try to understand how the heck they should proceed with this, answer lots of irrelevant questions and then in the end get the money in cash in my hand. (A fun side-note: Western Union’s SSL certificate has the wrong Common Name field in it. Certainly not the best way to give a good and trusty impression.)

To and from Africa we get to hear about “westernunion-like” businesses that tend to end up being blamed for sponsoring terror organizations. Perhaps we can somehow forgive parts of Africa where poverty is wide-spread and the internet/technological penetration is not so dense.

Within Sweden – I actually got an actual physical “100 kronor” bill mailed to me in an envelope the other day as a donation for curl. Of course I appreciate it, but I find it sad that sending an actual physical letter is the easiest and cheapest way!

Paypal is every so often just the most sensible and easiest solution out there. I’m just always so surprised that there are so few actual and serious competitors to Paypal – I mean that tries to function internationally. We have a handful Swedish or European-only style versions, but hey within Sweden and Europe SWIFT/IBAN rocks anyway so there’s little use for such a limited Paypal-clone.

I would guess that anyone who would seriously try to answer my questions would bring up legislations (and especially economy laws in various countries with money-laundry preventions and what not) and the conservatism of the banks that make big money by carving off a large amount or percentage on every transfer – not to mention the interest on the money during the insane periods when the money is in limbo somewhere between banks.

I would still rather like to see more companies and banks seriously trying to compete with Paypal about doing smooth payments from wherever to wherever. We’re in the 21st century darnit!

Electronics, Linux, Open Source

Neuros OSD 2.0

For you who are into things like open source hardware for your videos, it can be interesting to note Neuros‘ recent posting of their planned specs for their upcoming OSD 2.0 player that I guess then will replace the current Neuros OSD model.Neuros OSD 2.0

In hard techy interesting terms: they plan to upgrade to Texas Instruments Davinci 6446 chipset, which is a 300MHz ARM9 with a C64x DSP core embedded. Pretty much like the existing DM320 one, but it seems with a great deal of more horse power under the hood. Given their specs paper, it will support a lot of formats and at least partially up to HD resolutions. It’ll also support internal harddrive and offer 256MB RAM and 256MB internal NAND flash.

Personally I don’t care that much as I don’t even have analogue TV and don’t download/have many movies to watch and my existing DVB-T box has fine recording abilities and my DVD is good enough for my kids to repeatedly watch the same animated films over and over and over…

Oh btw, if this sounds like your kind of backyard and other things combine well, Neuros is hiring Linux developers for what I believe is this hardware.

(sorry for the crappy quality of the pic but I nicked it from the PDF)

Open Source, Technology

Snaxx v18

Me and my fellows of the Haxx tribe have been arranging the Snaxx series of gatherings for about eight years by now and on this Wednesday the 19th of March 2008 it is time for the 18th meeting. We gather up in a dark pub in Stockholm to talk tech, work, open source and similar crap with similar-minded people while going through the beers of the place.

Since several of us in Haxx are pretty much into ales, porters and stouts we tend to prefer places with a large selection of that kind of beer, both on draft and on bottle.

We tend to be somewhere around 10 to 15 friends, but I really prefer if you mail me and tell me if you intend to come so that we can arrange properly to deal with the amount of people that show up. We’re basically all friends and friends of friends.

So if you’re a friend and in Stockholm, welcome!

Electronics, Rockbox

Secrets are hard to keep

On the web, it seems more or less everything is eventually getting out and gets accessible for people to download and see. You can fight hard and make a real effort but as we keep seeing, lots of secrets get revealed in the end anyway.Data Sheet for a technical thing

The other day a friendly fellow joined the Rockbox IRC channel and pointed out a russian site full of iriver goodies such as service manuals, firmwares and perhaps most importantly: circuit diagrams for just about all (somewhat older) iriver models, including the H10, H100, H300 series and more.

I wish we had these before when the Rockbox ports for these targets were younger, but I’m sure they’ll still come handy to some people now as well.

(Now I won’t reproduce the link here just to not make the traffic too hard on that site and to avoid getting into problems for linking to content that is clearly dubious legal-wise, but you don’t have to be a genius to find the link yourself.)

Rockbox

Rockbox on iAudio M3

iAudio M3This player’s been out for quite some time and most certainly won’t be found anywhere to be bought as new. Still, Jens Arnold’s been working hard on this port lately and just minutes ago he mentioned that he got sound working on it – and the build table subsequently has all three kinds of the M3 build added to it.

The M3 is roughly an M5 but without any LCD on the main unit, there’s only that one on the LCD! And of course the M5 was almost an X5 but with a greyscale LCD… So it is another one of them Coldfire 5249 based targets. The LCD on the remote is 128×96 2bit greyscale,

The M3 comes with 20 or 40GB harddisk and there’s an M3L around with a larger capacity battery

How many Rockbox ports are there now? It’s hard to tell as it really depends on what level of maturity and how much functionality you require before you count it as an actual port, but the configure script in our source tree has 38 different ones mentioned. I believe some of them are not more than embryos, but there are also other initial efforts not yet added to this script.

Cowon’s spec page on the M3.

Rockbox

Rockbox on Meizu M6

Meizu M6In the eternal chase for new targets to port Rockbox to, the turn seems to have come to the tiny Meizu M6 player.

This 55 gram thing is slightly smaller than a credit card (width and height at least) and it boasts a 2.4″ LCD, 4GB flash and is powered by a Samsung SA58700 (ARM940T core and a CalmRISC16 DSP thing). It has an FM tuner and built-in mic for recording as well.

There’s of course the standard Rockbox forum thread,and an HW info page in the Rockbox wiki.

Other targets with the exact same SoC include the irivers E10, clix and S10. But none of those have a Rockbox port yet.

cURL and libcurl

curl feature freeze March 20 2008

It is yet again time to pause the add-new-features-craze in order to settle down and fix a few more remaining bugs before we go ship another curl and libcurl release in the beginning of April.

cURL

So at March 20 we hold back and only fix bugs for about 2 weeks until we release curl and libcurl 7.18.1.

The only currently mentioned flaw in TODO-RELEASE to fix before this release is the claimed race condition in win32 gethostbyname_thread but since the reporter doesn’t respond anymore and we can’t repeat the problem it is deemed to just be buried and forgotten.

Other problems currently mentioned on the mailing list is a POST problem with digest and read callbacks and a mysterious bad progress callbacks for uploads, but none of them seem very serious and thus terribly important to get fixed in case they should turn out hard-to-fix.

Yes, I picked the date on purpose as that is the magic date in this project. Especially this year.