Tag Archives: cURL and libcurl

cURL and libcurl

curl needs a fresh take on command line options

1 Comment

I just posted about this on the curl-users mailing list and I’ll just echo it here to reach a slightly larger audience:

One of the not so good behaviors of curl is how many of the command line options work when being repeated: toggling on/off.

We’ve got bug reports about this in the past and I know for a fact that this behavior has burnt more than one guy who’s tried to set default options for curl in their .curlrc etc. When they then re-use the same option on the command line or in a script, it effectively disables the option again…

I’d like this corrected. I want people to be able to explicitly enable and disable features with the command line options. I think the toggling is very rarely useful and something we can just abandon – unless we can figure out a way to keep it for backwards compatibility when we introduce the new behavior.

I’m willing to sacrifice some backwards compatibility to get this done, but I would of course like to hurt as few users as possible.

I’m very interested to get ideas and feedback from you guys on how we can accomplish this!

My first thoughts on how to do this, is simply to convert all the current options to enable options and then introduce a new concept that negates the option. Like -v or –verbose to enable verbose, and –no-verbose to disable verbose.

Any bright ideas?

Update: my suggestion above is what has now been committed targeted for the upcoming 7.19.0 release…

cURL and libcurl, Electronics

curl by sony

I got the latest Linux Journal issue (#170, June 2008) the other day and while reading through it I fell over the article about a guy who found a GPL license agreement among the papers for his brand new Sony TV.

… and there he also mentioned that they use curl! Fun! Apparently in their “Bravia Internet Video Link” product. Whatever that is… “Stream, browse and watch internet video and much more on your compatible BRAVIA® HDTV with the BRAVIA® Internet Video Link. It’s like extra channels for your new HDTV.“

Development, Open Source, Security

Coverity’s open source bug report

The great guys at scan.coverity.com published their Open Source Report 2008 in which they detail findings about source code they’ve monitored and how quality and bug density etc have changed over time since they started scanning over 250 popular open source projects. curl is one of the projects included.

Some highlights from the report:

  • curl is mentioned as one of the (few) projects that fixed all defects identified by coverity
  • from their start, the average defect frequency has gone down from one defect per 3333 lines of code to one defect per 4000 lines
  • they find no support to backup the old belief that there’s a correlation between function length and bug count
  • the average function length is 66 lines

And the top-5 most frequently detected defects are:

  1. NULL Pointer Dereference 28%
  2. Resource Leak 26%
  3. Unintentional Ignored Expressions 10%
  4. Use Before Test (NULL) 8%
  5. Buffer Overrun (statically allocated) 6%

For all details and more fun reading, see the full Open Source Report 2008 (1MB pdf)

cURL and libcurl, Open Source

libev vs libevent

Only a few years ago (autumn 2005) I was awarded a grant by iis.se (“The Internet Infrastructure Foundation” in Sweden) and in my subsequent work on the code for that project, I used libevent and implemented a high performance API for libcurl when dealing with very many simultaneous transfers.

Recently when I’ve read about people using the curl_multi_socket() API, I’ve seen mentions of the libev library and today when I’ve finally read up on the subject I fell over their very interesting performance comparison document comparing libev with libevent, including charts and all. Perhaps not a surprise when coming from one of the main the libev authors, but it seems libev does perform from some to significantly better than libevent depending on the circumstances.

When I get some time over I think I’ll try to port some of the example source codes over to use libev and see how it plays for me.

Update: I like how libev.com is used by Long Island Beverage Systems Inc… 🙂

cURL and libcurl, Licensing

The curl and the PHP

We have a sort of symbiosis between the curl project and the PHP project, at least we in the curl project get a lot of people learning about curl the first time when they hack PHP. This happens to the extent that to a lot of people, curl is but the name of a PHP extension.

So while we can thank the PHP project for referring us a bunch of users that might not otherwise have found us, there is also quite some “friction” or perhaps better called “disagreements” between our projects and how we (don’t) interact.PHP logo

name

CURL vs libcurl vs cURL. We only ever use the funny casing cURL when referring to the cURL project. The cURL project produces curl and libcurl. curl is a command line tool and libcurl is a file transfer library.

The PHP team provides and distributes an extension they call CURL which is a libcurl binding for PHP. This naming causes a great deal of confusion to PHP users who go to the curl site only to find that it isn’t at all devoted to (just) the PHP extension but instead there’s mostly a lot of other curl stuff there!

I’ve discussed this naming issue with the PHP team on several occasions but they don’t agree with me that this causes confusion, and even if it would cause confusion they seem to be of the opinion that it doesn’t matter since the PHP users should find all their info about CURL and related matters on the PHP site and thus it doesn’t matter what the curl site shows or not. (Or something similar to that, I really don’t mean to put words into their mouths so you better ask them about this to get their real and unaltered view – see my link to an old conversation for some info.)

I tend to call it PHP/CURL just to make sure it is clear that we’re talking about the binding. This of course also confuse users since that’s not what it is called in the PHP documentation…

irony

PHP themselves recognize the problem of related projects borrowing the name, so they forbid derivate projects to include “PHP” in their names. Clearly stated in paragraph 4 of their license.

versions

The binary build of PHP for windows have libcurl built in statically with the curl extension code, so people can’t easily replace the libcurl version used by PHP. And in general, Windows people using open source are much less likely to ever build anything on their own in my experience.

PHP 5.2.6 was released on May 1st 2008 still has libcurl 7.16.0 built into the Windows version. That libcurl version was released in October 2006 and right now we have released eight (8) releases after that one. All of them including many bug fixes. This is more than slightly annoying.

support

This isn’t anyone’s fault but… there really aren’t many PHP people who are involved or care about the libcurl binding so those who have PHP/CURL problems tend to ask questions on the curl-and-php mailing list and in the #curl IRC channel but there aren’t any PHP insiders around in those areas to answer PHP questions…

development

Is it just my imagination or isn’t there a lot of PHP users that have asked for the same features in the PHP libcurl binding for a long time by now, but really very few actually step forward and make a difference? So these features remain unfixed and not added. This is even “just” a binding, nothing of the really hard work is done in the binding itself… It might just be me and my head, but the ratio for doers/plain users in the PHP world seems to be exceptionally low in comparison to many other open source areas I see. Of course this is tainted by me only really seing the PHP/CURL side of the PHP world.

future

I have no reason to expect anything to change, nor do I know how I can make anything of this change on my own so I assume things will just continue working exactly like this in the future as well…

cURL and libcurl

libcurl send/recv functions

libcurl’s two newly added functions curl_easy_send() and curl_easy_recv() add a new way of working with data in libcurl. These two are just sending and receiving whatever on a connection previously setup with curl_easy_perform() with the CONNECT_ONLY option set.

I haven’t yet made the web site display the man pages for these functions, as the patch for them was just committed minutes ago. Update: the man pages is now added and this post has the links.

Get a feel for how it can work by reading the sendrecv.c example source.

cURL and libcurl

curl on z/os, symbian and os21

The curl project continues to conquer new markets and it continues to get ported to more platforms and operating systems. Just recently there have been reports about…

  1. A port to the IBM z/OS system, with the official IBM info about it being found here.
  2. Dan Fandrich ported it to the Symbian OS
  3. Christian Vogt mentioned that he had no troubles porting and using it on OS21

I’m trying to maintain a list of all CPUs and operating systems we have known ports being run on and if you have curl and/or libcurl running on another than the ones I list at the bottom of the INSTALL document, please let me know!

cURL and libcurl, Rockbox

More Means Less

1 Comment

Less is more it is said, and I can certainly subscribe to the reverse: more means less. The two primary open source projects I spend time in have been growing the last years, in source code contributions, but also in amount of users and in amount of contributors. I see the similar effects on myself and on my own role in both Rockbox and curl: I do more and more coordination, planning, admin work, talk (chatting on IRC, responding to mails etc) and “guidance” than actual coding work. My code/non-code work ratio has decreased massively.

This is not complaint, just an observation!

It makes sense to me that early on in a project, and until there’s enough momentum to get the project to more or less drive itself, it is important with a driving core that pushes the project forward. That makes sure every little peace fits together and gets the proper attention to make it a good product and project. As time goes by, more and more people get that knowledge, that ability and the amount of people that drive the project forward increases.

So being an “elderly” in both these projects, I’m more of an advisor, talker, tinker, admin, than a lead programmer now. This is at least most notable in Rockbox, since we have 80 committers now and I think at least 50 of them are active.

I probably spend roughly the same amount of time: somewhere around 2-3 hours/day on my open source projects.

Of course, in my particular case exactly now, I’ve also just recently ramped up my working hours and find myself trying to get accustomed to this life with full-time work, a two-kids-and-wife family and several time-consuming spare time projects. It takes a great deal of juggling and less sleeping.

Nothing is forever so I’m certain my situation will change over time. I’m determined to continue hacking in both projects. And my juggling skills will improve…

cURL and libcurl

Why curl sticks with CVS

Occasionally people ask me or just mock me because we’re still using CVS in the curl project, even though there are much more compelling alternatives out there now. Subversion, git, Bzr, Mercurial, etc. I am even a contributor and committer in the Subversion project. (Although I’ll be the first to admit that I never committed much and the stuff I did was done many years ago.)

CVS just isn’t bad enough to warrant the work of a replacement. curl is a tiny project (source code wise) and while CVS has several flaws in how it is designed and works, those flaws never hurt us much. Basically the only one is the lack of rename support and that has no major impact on us.

On the contrary, CVS has the upside of being established and rock solid since many years so people on all sorts of platforms can use it and get the curl source code. This is important especially for our automated build-system which we try hard to find volunteers for to run automatically daily around-the-clock (the results and outputs are then mailed to our central autobuild master server that collects and presents them) and then those guys need to be able to checkout the code easily. Using more modern tools will make it harder since those aren’t available as widely as binary packages for as many (outdated) platforms as CVS is.

So curl sticks with CVS for now.