curl and/or libcurl related
I previously mentioned on the libcurl mailing list, that Mark Nottingham in the IETF HTTP Working Group has initiated the work on putting together an overview of all (interesting) existing HTTP implementations
Of course curl is included in the bunch, or rather libcurl, but I would also urge you all to step forward and provide further details on other implementations you worked on or know of!
I just wanted to drop a note saying that the biggest explanation for the silence and slowness of my blog the last couple of weeks have been my ongoing vacation, which is still going on for another two weeks.
Of course things are happening still, but due to my lack of computer time right now I tend to prioritize actually working with those things rather than posting here writing about the stuff I do/read/fix.
The Rockbox Steering Board vote is over, results will be published soon (spoiler: I’m voted in as one of the members). There’s a curl release coming up for August and we still have a few outstanding issues to fix. More on these topics later!
I spoke to Anthony Bryan from the metalink project over Skype the other day, and the 16 minute recorded interview was recently posted so I thought I’d just announce my local copy of the 14MB file.
The topics should be of no surprise to readers of my blog: me, curl, Rockbox and metalink basically.
I like user feedback and comments from people in projects I participate in – even those that I run or maintain myself. I value bug reports and I think no project can evolve without a fair amount of external input.
But they can also be annoying since when done in public places they tend to stick around. If they’re negative I can respond to them if posted in forums where that is possible and where I care about it, but sometimes they’re just “blurted” out in a way that I cannot respond to and that I cannot do anything about. And the review/comment/complaint will sit there to be watched by the world. Uncommented by me or anyone else thinking otherwise.
Let me point out the recent example that made me write this particular rant: user review on curl at ohloh.
I realize there’s nobody to blame and that this is the way of life and how things work and that everybody is entitled to publish their opinions and all that. It still doesn’t feel really good when you just don’t agree with them and they’re “against” one of your own babies.
I noticed the new site publicsuffix.org that has been setup by the mozilla organization in an attempt to list public suffixes for all TLDs in the world, to basically know how to prevent sites from setting cookies that would span over just about all sites under that “public suffix”.
While I can see what drives this effort and since we have the same underlying problem in curl as well, I have sympathy for the effort. Still, I dread “having to” import and support this entire list in curl only to be able to better work like the browsers in the cookie department. Also, it feels like a cat and mouse race where the list may never be complete anyway. It is doomed to lack entries, or in the worst case list “public suffixes” that aren’t any such public suffixes anymore and thus it’ll prevent sites using that suffix to properly use cookies…
There’s no word on the site if IE or Opera etc are going to join this effort.
Update: there are several people expressing doubts about the virtues of this idea. Like Patrik Fältström on DNSOP.
Just minutes ago I uploaded the curl and libcurl 7.18.2 package to the curl site. There are a few new changes that people might just like, but most importantly there are many bug fixes.
And by a happy coincidence, a bunch of #curl visitors (the irc channel on freenode) are going to meet up for lunch on tuesday next week (June 10th) in Stockholm, Sweden. If you’re a curl hacker or curl fan and in the proximity that day, feel free to get in touch and join us!
I just posted about this on the curl-users mailing list and I’ll just echo it here to reach a slightly larger audience:
One of the not so good behaviors of curl is how many of the command line options work when being repeated: toggling on/off.
We’ve got bug reports about this in the past and I know for a fact that this behavior has burnt more than one guy who’s tried to set default options for curl in their .curlrc etc. When they then re-use the same option on the command line or in a script, it effectively disables the option again…
I’d like this corrected. I want people to be able to explicitly enable and disable features with the command line options. I think the toggling is very rarely useful and something we can just abandon – unless we can figure out a way to keep it for backwards compatibility when we introduce the new behavior.
I’m willing to sacrifice some backwards compatibility to get this done, but I would of course like to hurt as few users as possible.
I’m very interested to get ideas and feedback from you guys on how we can accomplish this!
My first thoughts on how to do this, is simply to convert all the current options to enable options and then introduce a new concept that negates the option. Like -v or –verbose to enable verbose, and –no-verbose to disable verbose.
Any bright ideas?
Update: my suggestion above is what has now been committed targeted for the upcoming 7.19.0 release…
I got the latest Linux Journal issue (#170, June 2008) the other day and while reading through it I fell over the article about a guy who found a GPL license agreement among the papers for his brand new Sony TV.
… and there he also mentioned that they use curl! Fun! Apparently in their “Bravia Internet Video Link” product. Whatever that is… “Stream, browse and watch internet video and much more on your compatible BRAVIA® HDTV with the BRAVIA® Internet Video Link. It’s like extra channels for your new HDTV.“
Only a few years ago (autumn 2005) I was awarded a grant by iis.se (“The Internet Infrastructure Foundation” in Sweden) and in my subsequent work on the code for that project, I used libevent and implemented a high performance API for libcurl when dealing with very many simultaneous transfers.
Recently when I’ve read about people using the curl_multi_socket() API, I’ve seen mentions of the libev library and today when I’ve finally read up on the subject I fell over their very interesting performance comparison document comparing libev with libevent, including charts and all. Perhaps not a surprise when coming from one of the main the libev authors, but it seems libev does perform from some to significantly better than libevent depending on the circumstances.
When I get some time over I think I’ll try to port some of the example source codes over to use libev and see how it plays for me.
Update: I like how libev.com is used by Long Island Beverage Systems Inc… 🙂
We have a sort of symbiosis between the curl project and the PHP project, at least we in the curl project get a lot of people learning about curl the first time when they hack PHP. This happens to the extent that to a lot of people, curl is but the name of a PHP extension.
So while we can thank the PHP project for referring us a bunch of users that might not otherwise have found us, there is also quite some “friction” or perhaps better called “disagreements” between our projects and how we (don’t) interact.
name
CURL vs libcurl vs cURL. We only ever use the funny casing cURL when referring to the cURL project. The cURL project produces curl and libcurl. curl is a command line tool and libcurl is a file transfer library.
The PHP team provides and distributes an extension they call CURL which is a libcurl binding for PHP. This naming causes a great deal of confusion to PHP users who go to the curl site only to find that it isn’t at all devoted to (just) the PHP extension but instead there’s mostly a lot of other curl stuff there!
I’ve discussed this naming issue with the PHP team on several occasions but they don’t agree with me that this causes confusion, and even if it would cause confusion they seem to be of the opinion that it doesn’t matter since the PHP users should find all their info about CURL and related matters on the PHP site and thus it doesn’t matter what the curl site shows or not. (Or something similar to that, I really don’t mean to put words into their mouths so you better ask them about this to get their real and unaltered view – see my link to an old conversation for some info.)
I tend to call it PHP/CURL just to make sure it is clear that we’re talking about the binding. This of course also confuse users since that’s not what it is called in the PHP documentation…
irony
PHP themselves recognize the problem of related projects borrowing the name, so they forbid derivate projects to include “PHP” in their names. Clearly stated in paragraph 4 of their license.
versions
The binary build of PHP for windows have libcurl built in statically with the curl extension code, so people can’t easily replace the libcurl version used by PHP. And in general, Windows people using open source are much less likely to ever build anything on their own in my experience.
PHP 5.2.6 was released on May 1st 2008 still has libcurl 7.16.0 built into the Windows version. That libcurl version was released in October 2006 and right now we have released eight (8) releases after that one. All of them including many bug fixes. This is more than slightly annoying.
support
This isn’t anyone’s fault but… there really aren’t many PHP people who are involved or care about the libcurl binding so those who have PHP/CURL problems tend to ask questions on the curl-and-php mailing list and in the #curl IRC channel but there aren’t any PHP insiders around in those areas to answer PHP questions…
development
Is it just my imagination or isn’t there a lot of PHP users that have asked for the same features in the PHP libcurl binding for a long time by now, but really very few actually step forward and make a difference? So these features remain unfixed and not added. This is even “just” a binding, nothing of the really hard work is done in the binding itself… It might just be me and my head, but the ratio for doers/plain users in the PHP world seems to be exceptionally low in comparison to many other open source areas I see. Of course this is tainted by me only really seing the PHP/CURL side of the PHP world.
future
I have no reason to expect anything to change, nor do I know how I can make anything of this change on my own so I assume things will just continue working exactly like this in the future as well…