Category Archives: cURL and libcurl

curl and/or libcurl related

cURL and libcurl, Network, Open Source

murl for extended curlness

1 Comment

I’m a firm believer in the old unix mantra of letting each tool do its job and do it well, and pass on the rest of the work to the next tool. I’ve always stated that curl should remain this way and that it should remain within its defined walls and not try to do everything.

But time passes and more and more ideas are thrown up in the air, or in some cases directly at me, and the list of things that we could do but don’t due to this philosophical limit of remaining focused has grown. It currently includes at least:

  • metalink support
  • recursive HTML downloads
  • recursive/wildcard FTP transfers
  • bittorrent support
  • automatic proxy configuration
  • simultaneous/parallel download support

Educated readers of course immediately detect that this list (if implemented) would make a tool that basically does what wget already does (and a lot more) and I’ve explicitly said for a decade that curl is not a wget clone. Maybe it is time for us (me?) to reevaluate that sentiment – at least in some sense.

I don’t want to sacrifice the concepts that have worked so fine for curl under so many years, so I’m still firmly against stuffing all this into curl (or libcurl). That simply will not happen with me at the wheel.

A much more interesting alternative would be to instead start working on a second tool within the curl project: murl. A tool that does basically everything that curl already does, but also opens the doors for adding just about everything else we can cram in and that is still related to data transfers. That would include, but not be restricted to, all the fancy stuff mentioned in the list above!

No the name murl is not set in stone, nor is this whole idea anything but plain and early thoughts thrown out at this point so it may or may not actually take off. It will probably depend on if I get support and help from fellow hackers to get started and moving along.

cURL

cURL and libcurl, Security

curl 7.19.4

curl and libcurl 7.19.4 has just been released! This time I think the perhaps most notable fix is the CVS-2009-0037 security fix which this release addresses. A little over 600 days passed since the previous vulnerability was announced.

Other than that major event, there are a bunch of interesting changes in this release:

  • Added CURLOPT_NOPROXY and the corresponding –noproxy
  • the OpenSSL-specific code disables TICKET (rfc5077) which is enabled by default in openssl 0.9.8j
  • Added CURLOPT_TFTP_BLKSIZE
  • Added CURLOPT_SOCKS5_GSSAPI_SERVICE and CURLOPT_SOCKS5_GSSAPI_NEC – with the corresponding curl options –socks5-gssapi-service and –socks5-gssapi-nec
  • Improved IPv6 support when built with with c-ares >= 1.6.1
  • Added CURLPROXY_HTTP_1_0 and –proxy1.0
  • Added docs/libcurl/symbols-in-versions
  • Added CURLINFO_CONDITION_UNMET
  • Added support for Digest and NTLM authentication using GnuTLS
  • CURLOPT_FTP_CREATE_MISSING_DIRS can now be set to 2 to retry the CWD even when MKD fails
  • GnuTLS initing moved to curl_global_init()
  • Added CURLOPT_REDIR_PROTOCOLS and CURLOPT_PROTOCOLS

We also did at least 15 documented bugfixes in this release and 25 people are credited for their help to make it happen.

cURL and libcurl

Saying lib out loud

9 Comments

Not being a native English speaker, I’ve always pronounced libcurl with a ‘lib’ part as if it was part of ‘liberty’, and ‘curl’ with a K sound and ending with ‘earl’. I don’t know of any Swedes who would not pronounce ‘lib’ like that, but when speaking Swedish we’re of course highly influenced by other things so it’s not really relevant.

It wasn’t until I got on FLOSS Weekly that I fully realized that some English speakers would actually pronounce the ‘lib’ part as the first syllable of ‘library’ and that does make sense considering lib is short for library.

But libcurl is just a smaller player here. How do you English speakers pronounce libc? libxml? libgcc?

And yes, this is another one of those really important issues in life. Almost as important as how to close parenthetical expressions with emoticons!


cURL and libcurl, Development

Some stats on curl development

Counting curl 6.0 and up to curl 7.19.3 we’ve done 78 releases during the 9.4 years it took.

In this time, we’ve mentioned 1259 bugfixes and 389 notable changes.

This makes one bugfix done every 2.7 days. One release done every 43rd day with an average of 16 bugfixes done in each. The longest interval ever between two curl releases was 139 days, back in 2000 when we worked to release the first version 7 release (known as 7.1).

To compare with how our work has been more recently, doing the same math limited to the 20 latest releases only (the 3.3 years since and including 7.15.0) shows that we’re still on 2.7 days per bugfix (although we know that the code base has grown steadily for years) but we’re now on 61 days between releases and 21 bugfixes/release…

All this info and more will be visible on a web page on the curl site soonish, I’m still working on polishing it up.

What other useful or useless but interesting numbers could be extracted from this?

cURL and libcurl

curl 7.19.3

I just now sent away the announcement of curl and libcurl 7.19.3. With some 30 bugfixes and only two actual changes I hope this will again be a solid release that’ll be appreciated and used all over.

The changes are:

  • CURLAUTH_DIGEST_IE bit added for CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH – as older Internet Explorers have an “interesting” take at the Digest authentication and servers that speak that dialect doesn’t like libcurl’s regular way
  • VC9 Makefiles were added to the release package, for the VS2008 users of the world

Download here.

cURL and libcurl, Linux, Security

Linux distros consolidate crypto libs

For a while already, the Fedora distribution has fought battles, done lots of work and pushed for a consolidation of all packages that use crypto libs to completely go with Mozilla’s NSS.

Now it seems to be OpenSUSE’s turn. The discussion I link to here doesn’t make any definite conclusions but they seem to lean towards NSS as well, claiming it has the most features. I wonder what they base that statement on – if there’s a public doc anywhere that state exactly which has what that makes any contender better than any other for them?

In the Fedora case it seems they’ve focused on the NSS FIPS license as the deciding factor but the license issue is also often brought up in this discussion.

I’ve personally been pondering on writing some kind of unified crypto layer that would expose a single API to an application and handle the different libs as backends, pretty much the same way we do it internally in libcurl at the moment. It hasn’t taken off (or even been started) since I’ve not had the time nor energy for it yet.

cURL and libcurl

FLOSS Weekly #51 on curl

4 Comments

FLOSS WeeklyLate Wednesday evening (middle European time zone) on January 7th 2009 I was up doing a live recording of the podcast show FLOSS Weekly with Leo Laporte and Randal Schwartz. This recording is now available for download as episode #51.

We chatted a bit about curl and libcurl and I think I did a decent job of keeping to the subject and not making a total fool of myself. Enjoy!

(The talk was done using skype and yes my laptop was running Windows at the time…!)

cURL and libcurl, Development, Linux, Network, Web

IETF http-state group created

Over at the IETF another group was just created named http-state (with an associated mailing list) with the specific goal:

Ultimately, the purpose of this group is to create an updated HTTP State Management Mechanism RFC (aka cookies) that will supersede the Netscape spec, RFCs 2109, 2964, 2965 then add in real-world usage (e.g. HTTPOnly), and possibly add in additional features and possibly merge in draft-broyer-http-cookie-auth-00.txt and draft-pettersen-cookie-v2-03.txt.

I’ve joined the list and I hope to follow and participate in this, as I believe the current state of HTTP cookies is a rather sorry mess and the Netscape spec is still what closest describes how cookies work in the wild. Of course I’ll do it with my libcurl experience in my luggage.

While it perhaps would be cool to join the group in more formal way, there’s no way for me to participate in that IETF meeting in San Francisco in March.

cURL and libcurl, libssh2

A new year with new fun

2 Comments

I had a great and relaxing winter/Christmas holiday and hence my silence here.

I’m now back up to speed, with a podcast interview done yesterday (I’ll post another entry when it gets available), I do some funded development on libcurl and libssh2 in the background while I’m spending my days at my client’s place working on a 10G traffic analyzer product.

It was rather calm during “the break” but I’ve now noticed that at least the curl project has gotten significantly increased activity again. We’re on a feature freeze now for the January release, but there seems to be at least 4 patches pending adding new stuff for the release planned to come after this (around March if things go well).