Tag Archives: cURL and libcurl

cURL and libcurl

curl and libcurl 7.18.1

Mainly thanks to the 22 friends named in the release notes, curl and libcurl 7.18.1 was released today with the news and fixes that should prove this the best curl and libcurl versions ever – I guess we always have to believe that our latest is the greatest, why else would we release it?

cURL

The release notes identifies 23 bug fixes we did during the two months since the last release, and the news we introduce include these goodies:

  • added support for “HttpOnly” cookies
  • ‘make ca-bundle’ downloads and generates an updated ca bundle file
  • we no longer distribute or install a ca cert bundle
  • SSLv2 is now disabled by default for SSL operations
  • the test509-style setting URL in callback is officially no longer supported
  • support a full chain of certificates in a given PKCS12 certificate
  • resumed transfers work with SFTP
  • added type checking macros for curl_easy_setopt() and curl_easy_getinfo(), watch out for new warnings in code using libcurl (needs gcc-4.3 and currently only works in C mode)
  • curl_easy_setopt(), curl_easy_getinfo(), curl_share_setopt() and curl_multi_setopt() uses are now checked to use exactly three arguments
  • –with-ca-path=DIR configure option allows to set an openSSL CApath instead of a default ca bundle.

As usual, you can download it here.

cURL and libcurl, Rockbox

Two for the price of one

As probably one of the last humans on earth, I got my paper version of Linux Journal’s April 2008 issue in my mailbox today. While I’ve felt that the magazine has slowly degraded over the last years, this issue certainly made me all warm and fuzzy inside since it contains not one but two articles on subjects very dear to me. A three-page article was dedicated to a Rockbox review (subscription-required-to-read link) named “iPod + Rockbox =Linux Journal April 2008 issue 168 Entertainment Extravaganza” and there was a two-page article on how to use curl (another paid-subscription-required link)to fiddle with a web interface named “Twittering from the Command Line”.

I have a hard time to put a finger on exactly why I feel that the magazine has lost some of its past qualities but I’ve noticed that I finish the magazine much faster these days and I generally skip over more uninteresting articles. I haven’t really investigated this, but it may very well be because a shift in focus in the magazine. I tend to be interested in low-level, kernel, OS, hardware-touching, libraries and related stuff. Not so much in high-level applications or web frameworks and more user and desktop oriented fluff.

I should also add that the Rockbox review was pretty good. Not perfect in any way since it contained errors, was for an old boring grayscale-ipod target and suggested that the original OS is better suited for playing music… The article that mentioned curl wasn’t really about curl but only showed how it can be used but it was mentioned in positive terms at least!

cURL and libcurl

curl ten years today

1 Comment

Birthdaycake

On March 20th 1998 curl 4 was released. It was the first curl release ever even if already at version 4 since we kept the version number from the previous projects we did before curl – using other names. We started it all with having the tool named httpget (which was an existing small tool written by Rafael Sagula), soon changed name to urlget to end up with curl – all renames happening due to shifting features and focus.

Like many other projects, this started because of an itch. I wanted to get currency rates off the internet to allow an IRC bot to be able to provide an “exchange service” for users with accurate up-to-date rates. I thought the existing projects I found all did too much or did the wrong thing. That bot and service is now gone since long.

curl has been a truly portable project from day 1, and the first windows build was already urlget 2.1 (pre-curl). autoconf support for the build process was added in October 1998.

Unfortunately I don’t have the original release 4 tarball left anymore, the closest one I have is curl 4.8 (dated August 31 1998). curl 4.8 is about 3400 lines of code. Today we’re totaling in well over 100K source lines, so it has grown over 30 times!

I had no big plans for curl nor did I think very much about the future of the project. I just added the features I and my fellow contributors wanted to have for the moment. That’s actually pretty much how the project has continued to work. We don’t have many long-term plans for what to do with it, we mostly look just inches ahead of our noses and act accordingly.

During the version 6 period (Sep 1999 – Mar 2000) we learned that curl was getting popular, was useful and worked rather well, so the work on providing a libcurl started. We wanted to offer other applications the ability to use curl’s file transfer powers. Version 7.1 was released in August 2000 and thus libcurl was officially born.

curl and libcurl remained being a rather low-key project, I just work on it on my spare time and there are no full-time developers paid to work on this project – apart from some occasional sub-projects now and then that have been sponsored by companies and organizations. (See later on for an example.)

Slowly but surely more and more people started using libcurl and contributed with bug reports and patches. When the project turned 5 years in 2003 I collected all the names of all contributors so far and I reached the number 270. I found the number very high and I was mostly kidding when I said I hoped we would double that amount by the time we celebrate our tenth anniversary. Of course we’ve more than doubled that amount today when we have more than 620 named contributors so far – and continuously adding new ones with every release.

During this journey of a decade, I’ve remained the lead developer and project leader but we’re now some 10 developers with commit access (that also use it) and I try to be open and responsive in order to attract more developers to come aboard, to listen to their advice and ideas and to be sensitive on what our users want from us.

In 2005 I was lucky enough to get a grant from the Swedish IIS organization for the purpose of developing a new event-based API for libcurl to better deal with very large amount of connections, the problem so nicely called c10k.

In the days when our humble project turns 10, I spend about two hours spare time per day on the project and it is my primary hobby, we make 5-6 releases per year, we get about 7000 unique visitors on the web site a normal day, about one million curl packages are downloaded per year – from our servers.

Today, libcurl is feature-rich, portable, very widely used, very fast, well supported and there are no signs of stagnation in release nor development pace. In fact, looking at the source-code growth over the last couple of years we can see a pretty stable and continuous growth:

curl source code growth

Just as I never looked ahead and planned for the future much in the past, I don’t do that now either so I really don’t know and can’t tell what the future will hold for us. We’ll just continue to develop the world’s best client-side file transfer library, to make it even more solid for the foreseeable future, to make it do the things users and developers out there think it should do. Possibly that involves adding support for more protocols, removing some of the less popular ones or simply by enhancing how we support the existing ones.

Join the mailing lists and join us for the next ten years to come!

cURL and libcurl

curl feature freeze March 20 2008

It is yet again time to pause the add-new-features-craze in order to settle down and fix a few more remaining bugs before we go ship another curl and libcurl release in the beginning of April.

cURL

So at March 20 we hold back and only fix bugs for about 2 weeks until we release curl and libcurl 7.18.1.

The only currently mentioned flaw in TODO-RELEASE to fix before this release is the claimed race condition in win32 gethostbyname_thread but since the reporter doesn’t respond anymore and we can’t repeat the problem it is deemed to just be buried and forgotten.

Other problems currently mentioned on the mailing list is a POST problem with digest and read callbacks and a mysterious bad progress callbacks for uploads, but none of them seem very serious and thus terribly important to get fixed in case they should turn out hard-to-fix.

Yes, I picked the date on purpose as that is the magic date in this project. Especially this year.

cURL and libcurl, Rockbox

Summer of code ideas and mentors!

To get good results from Google’s Summer of Code, we need a fair amount of volunteering mentors and we need a good set of interesting projects to make students get attracted.

Rockbox tinyIf your interest is in the Rockbox project, add your project ideas or add yourself as mentor on this wiki page.

curl tinyIf your interest is in the cURL project, read this page about the existing ideas and provide new ones or submit yourself as mentor on the mailing list!

Organizations can apply for becoming part of this starting tomorrow, March 3 2008.

cURL and libcurl, Licensing

AOL UK uses curl in disguise

2 Comments

Information to this was mailed to me from a friend but is easily verified as I’ll describe below.curl tiny

America Online in the UK (AOL UK) is using our cURL application (without including the license anywhere) as part of their automated broadband router configuration CD for their AOL UK customer base. The CD is provided to all AOL UK customers and the automated router configuration component using cURL has been included with it for a couple of years.

The software includes the cURL application renamed to “AOL_Broadband_Installer.EXE“. There is no license included or mention of the license anywhere on the CD or installer, contrary to what’s required at http://curl.haxx.se/docs/copyright.html.

The md5sum for AOL_Broadband_Installer.EXE matches the win32 binary in the curl-7.15.3-win32-nossl.zip release package I personally built and offer for download…!

If you want to check it out yourself, the direct link I figured out to the installer is here and I found it on this page (download the “easy installer” for Netgear DG834G).

Update: see my reply below.

cURL and libcurl, Rockbox

Google Summer of Code for cURL?

Google Summer of Code 2007 front print

As I was involved in gsoc 2007 within the Rockbox project, I ventilated the idea on the libcurl mailing list just yesterday that perhaps this is a good year for the cURL project to apply to become a mentoring organization to be able to host students doing gsoc work?

If so, this is no point unless we can at least present a bunch of interesting projects to lure students to us to have them work to improve (lib)curl and do stuff we otherwise might have a hard time to get done.

What things would you like to see that you consider would be a good project for a student to work on during the summer 2008?

New protocols? Fixing the last remaining blocking calls within libcurl? Fixing up/replacing language bindings? It’s not strictly a requirement that we come up with the best ideas since students apply with their own suggestion anyway, but we can provide good suggestions and ideas that will make students attracted to us and make them select to work for our project – should we be selected as a mentor organization.

cURL

cURL and libcurl

CA cert bundle or not

Since the dawn of time (at least it feels that long) we’ve included a copy of a ca cert bundle in the curl releases. That ca cert bundle originates from Netscape 4.72 and no cert has been added to it since the year 2000(!)

Instead, we’ve offered things like an easy downloadable version from our web site, and documented that this is what you often need to do.

Anyway, we were recently triggered by a bug report and are discussing updating the bundle in the curl tarballs – we’ll just need to sort out the license situation first but we’re slowly progressing there and I think we’re pretty fine with things as they are right now.

However, the question is perhaps better put the other way: why should we bother to include a ca cert bundle in the first place? Most users will already have one in their system (since basically all SSL-based applications want one) and those that don’t can very easily get an updated one using our online server or a recent perl script added to the curl source tree.

I hope I don’t have to tell you that I value all input I can get on this issue!

cURL and libcurl

Make Them Pick Us

Given that there are an endless series of open source and free software projects around. What makes companies and projects likely to chose to depend and use one of the existing ones rather than to write it themselves or possibly buy a closed-source solution instead? I’ll try to answer a few of the things that might matter, and deal with how curl and libcurl relates to them.

Proven Track Record

The project needs to have been around for a while, so that external people can see that the development continues and that there is a continued interest in the project from developers and users. That bug reports are acknowledged and fixed, that it has been scrutinized for the most obvious security problems etc. The curl project started almost ten years ago, have done more than one hundred releases and there is now more developer activity in the project than ever before.

Certified Goodness

With companies and associations that “certify” others, you can get others’ views on the quality of the projects.

The company named OpenLogic offers “certification” of open source software for companies to feel safer. I must admit I like seeing they’ve certified curl and libcurl. You can get their sales-pitch style description of their certification process here.

Of course I also like to see curl going to rung 2 on the scan.coverity.com list as it would mean a second (independent from the first) source would also claim that there’s a reasonable level of quality in the product.

If they did it so can we

With a vast list of existing companies and products that already are using the project, newcomers can see that this and that company and project already depend on this, and that fact alone makes the project even more likely to be a solid and trustworthy choice.

Being the answer when the question comes

Being known is important. When someone asks for help and guidance about what possible solutions there are to a particular problem, you want a large portion of your target audience to know about your project and to say “oh for doing X you could try project Y”. I want people to think libcurl when asked a question about doing internet-related transfers, like HTTP or FTP.

This is of course a matter of marketing and getting known to lots of people is a hard thing for an open source project with nothing but volunteers with no particular company backing.

Being a fine project

Of course the prerequisite to all points above is that the project is well maintained, the source is written in a nice manner and that there’s an open and prosperous community…