Tag Archives: cURL and libcurl

cURL and libcurl, Open Source

Getting support to curl

The other day I read this blog post by Stormy Peters, talking about getting people to sponsor or support Open Source projects and she continued to describe the Gnome approach and a bunch of projects that accept donations etc etc.

It made me (not too surprising) think about the situation for our little project cURL. We’re independent of any umbrella organization (GNU, ASF, etc) and we don’t have any vendor or company backing that pays for daily development or maintenance. We don’t have any legal entity or formal organization behind the project. We’re all just a bunch of people on some mailing lists.

We do have occasional companies and vendors who step up and pay individual developers to add features or provide various kinds of support, but they’re all basically single-shot occurrences and nothing that’s done on an ongoing basis.

Or products are used in all Linux distros, by hundreds of companies and so on. We’re a fairly active team, continuously working on bug fixes, tweaks and adding new features.

What can we do to make us more attractive for more support or active sponsoring by some vendor(s)?

Would joining an “umbrella” organization or forming a legal entity make it any more likely to happen?

Isn’t it so, that if the project is mature and good enough already, there’s actually very very little incentive for any company to take it under their wings and rather the market economy makes it a lot more profitable to simply use it as it is and if – at worst – in the end something really hits the fan, you can pay someone at that crisis point to fix up the immediate problem. And then continue like before.

And to be honest, I think we are proving to everyone that it works this way by continuing to deliver rock solid quality software. For no price. Completely open source. Year after year. Darnit, it’s just too fun to stop!

cURL

cURL and libcurl, libssh2

Adding known hosts support

… to libcurl and libssh2!

I’m about to start this little mini adventure, so if you’re one of the guys out there who’s been looking forward to be able to do even more (Open)SSH-like things with curl and libcurl when we use SCP and SFTP then consider this a little notification to start listening!

This will require improvements and changes in both projects, and funnily enough I’m already involved knee-deep in both so that shouldn’t cause any problems. I do however greatly appreciate feedback and reviews of my pending implementation proposals! I want this done in a way that benefits many and that isn’t too likely to break at least within the nearest future.

Ok, enough of that. Stand by for posts to the mailing lists. I’ll start off with the libcurl one which will thus be a slightly higher level API for all this. I’ll update this blog post later on to feature direct links to my proposals. Please consider posting responses to the suggestions to the appropriate mailing list!

The libcurl proposal

The first mail to libssh2-devel

cURL and libcurl, Linux, Mail, Network, Web

HTTP Status Report

1 Comment

Mark Nottingham Mark Nottingham held a very interesting one hour talk on the status of HTTP and the work on HTTPbis on a QCon conference recently, and luckily for us HTTP geeks there’s this great video/presentation from that.

curl is mentioned at least twice in the slides, unfortunately it has a wrong fact on the second mention where it says curl uses “Pragma: no-cache” as it isn’t true anymore. It used to do that, but we’ve stopped doing it in curl since a while ago.

I’m a subscriber to the httpbis mailing list and a casual contributor, but nonetheless his summary and overview of the state was refreshing as I’ve not been able to keep up with all the details and I haven’t been tracking that working group from its start either.

cURL and libcurl, Open Source

Code re-use is fun

Back in 2003 I wrote up support for the HTTP NTLM authentication method for libcurl. Happy with my achievement, I later that year donated a GPL licensed version of my code to the Wget project (which also was my first contact with the signed paper stuff with the GNU/FSF to waive my copyright claims and instead hand them over). What was perhaps not so amusing with this code was when both curl and Wget 2005 were discovered to have the same security flaw due to my mistakes in this code shared by both projects!

Just recently, the neon project seems to be interested in taking on the version I adjusted somewhat for them, so possibly the third HTTP code is soon using this. Yeah I posted it on their mailing list back then so it has been sitting there in the archives maturing for some 6 years by now…

I also happened to fall over the SSH Tunnel Creator tool, which I’ve never used myself, that apparently snatched my neon donation (quite according to what the license allowed of course) and used it in their tool to do NTLM!

It’s actually not until recent years I discovered libntlm, and while I don’t know how good it was back in the days when I wrote my first NTLM stuff I generally think using existing libs is the better idea…

cURL and libcurl, Network, Open Source

murl for extended curlness

1 Comment

I’m a firm believer in the old unix mantra of letting each tool do its job and do it well, and pass on the rest of the work to the next tool. I’ve always stated that curl should remain this way and that it should remain within its defined walls and not try to do everything.

But time passes and more and more ideas are thrown up in the air, or in some cases directly at me, and the list of things that we could do but don’t due to this philosophical limit of remaining focused has grown. It currently includes at least:

  • metalink support
  • recursive HTML downloads
  • recursive/wildcard FTP transfers
  • bittorrent support
  • automatic proxy configuration
  • simultaneous/parallel download support

Educated readers of course immediately detect that this list (if implemented) would make a tool that basically does what wget already does (and a lot more) and I’ve explicitly said for a decade that curl is not a wget clone. Maybe it is time for us (me?) to reevaluate that sentiment – at least in some sense.

I don’t want to sacrifice the concepts that have worked so fine for curl under so many years, so I’m still firmly against stuffing all this into curl (or libcurl). That simply will not happen with me at the wheel.

A much more interesting alternative would be to instead start working on a second tool within the curl project: murl. A tool that does basically everything that curl already does, but also opens the doors for adding just about everything else we can cram in and that is still related to data transfers. That would include, but not be restricted to, all the fancy stuff mentioned in the list above!

No the name murl is not set in stone, nor is this whole idea anything but plain and early thoughts thrown out at this point so it may or may not actually take off. It will probably depend on if I get support and help from fellow hackers to get started and moving along.

cURL

cURL and libcurl, Security

curl 7.19.4

curl and libcurl 7.19.4 has just been released! This time I think the perhaps most notable fix is the CVS-2009-0037 security fix which this release addresses. A little over 600 days passed since the previous vulnerability was announced.

Other than that major event, there are a bunch of interesting changes in this release:

  • Added CURLOPT_NOPROXY and the corresponding –noproxy
  • the OpenSSL-specific code disables TICKET (rfc5077) which is enabled by default in openssl 0.9.8j
  • Added CURLOPT_TFTP_BLKSIZE
  • Added CURLOPT_SOCKS5_GSSAPI_SERVICE and CURLOPT_SOCKS5_GSSAPI_NEC – with the corresponding curl options –socks5-gssapi-service and –socks5-gssapi-nec
  • Improved IPv6 support when built with with c-ares >= 1.6.1
  • Added CURLPROXY_HTTP_1_0 and –proxy1.0
  • Added docs/libcurl/symbols-in-versions
  • Added CURLINFO_CONDITION_UNMET
  • Added support for Digest and NTLM authentication using GnuTLS
  • CURLOPT_FTP_CREATE_MISSING_DIRS can now be set to 2 to retry the CWD even when MKD fails
  • GnuTLS initing moved to curl_global_init()
  • Added CURLOPT_REDIR_PROTOCOLS and CURLOPT_PROTOCOLS

We also did at least 15 documented bugfixes in this release and 25 people are credited for their help to make it happen.

cURL and libcurl

Saying lib out loud

9 Comments

Not being a native English speaker, I’ve always pronounced libcurl with a ‘lib’ part as if it was part of ‘liberty’, and ‘curl’ with a K sound and ending with ‘earl’. I don’t know of any Swedes who would not pronounce ‘lib’ like that, but when speaking Swedish we’re of course highly influenced by other things so it’s not really relevant.

It wasn’t until I got on FLOSS Weekly that I fully realized that some English speakers would actually pronounce the ‘lib’ part as the first syllable of ‘library’ and that does make sense considering lib is short for library.

But libcurl is just a smaller player here. How do you English speakers pronounce libc? libxml? libgcc?

And yes, this is another one of those really important issues in life. Almost as important as how to close parenthetical expressions with emoticons!