Tag Archives: cURL and libcurl

cURL and libcurl, Licensing

The curl and the PHP

We have a sort of symbiosis between the curl project and the PHP project, at least we in the curl project get a lot of people learning about curl the first time when they hack PHP. This happens to the extent that to a lot of people, curl is but the name of a PHP extension.

So while we can thank the PHP project for referring us a bunch of users that might not otherwise have found us, there is also quite some “friction” or perhaps better called “disagreements” between our projects and how we (don’t) interact.PHP logo

name

CURL vs libcurl vs cURL. We only ever use the funny casing cURL when referring to the cURL project. The cURL project produces curl and libcurl. curl is a command line tool and libcurl is a file transfer library.

The PHP team provides and distributes an extension they call CURL which is a libcurl binding for PHP. This naming causes a great deal of confusion to PHP users who go to the curl site only to find that it isn’t at all devoted to (just) the PHP extension but instead there’s mostly a lot of other curl stuff there!

I’ve discussed this naming issue with the PHP team on several occasions but they don’t agree with me that this causes confusion, and even if it would cause confusion they seem to be of the opinion that it doesn’t matter since the PHP users should find all their info about CURL and related matters on the PHP site and thus it doesn’t matter what the curl site shows or not. (Or something similar to that, I really don’t mean to put words into their mouths so you better ask them about this to get their real and unaltered view – see my link to an old conversation for some info.)

I tend to call it PHP/CURL just to make sure it is clear that we’re talking about the binding. This of course also confuse users since that’s not what it is called in the PHP documentation…

irony

PHP themselves recognize the problem of related projects borrowing the name, so they forbid derivate projects to include “PHP” in their names. Clearly stated in paragraph 4 of their license.

versions

The binary build of PHP for windows have libcurl built in statically with the curl extension code, so people can’t easily replace the libcurl version used by PHP. And in general, Windows people using open source are much less likely to ever build anything on their own in my experience.

PHP 5.2.6 was released on May 1st 2008 still has libcurl 7.16.0 built into the Windows version. That libcurl version was released in October 2006 and right now we have released eight (8) releases after that one. All of them including many bug fixes. This is more than slightly annoying.

support

This isn’t anyone’s fault but… there really aren’t many PHP people who are involved or care about the libcurl binding so those who have PHP/CURL problems tend to ask questions on the curl-and-php mailing list and in the #curl IRC channel but there aren’t any PHP insiders around in those areas to answer PHP questions…

development

Is it just my imagination or isn’t there a lot of PHP users that have asked for the same features in the PHP libcurl binding for a long time by now, but really very few actually step forward and make a difference? So these features remain unfixed and not added. This is even “just” a binding, nothing of the really hard work is done in the binding itself… It might just be me and my head, but the ratio for doers/plain users in the PHP world seems to be exceptionally low in comparison to many other open source areas I see. Of course this is tainted by me only really seing the PHP/CURL side of the PHP world.

future

I have no reason to expect anything to change, nor do I know how I can make anything of this change on my own so I assume things will just continue working exactly like this in the future as well…

cURL and libcurl

libcurl send/recv functions

libcurl’s two newly added functions curl_easy_send() and curl_easy_recv() add a new way of working with data in libcurl. These two are just sending and receiving whatever on a connection previously setup with curl_easy_perform() with the CONNECT_ONLY option set.

I haven’t yet made the web site display the man pages for these functions, as the patch for them was just committed minutes ago. Update: the man pages is now added and this post has the links.

Get a feel for how it can work by reading the sendrecv.c example source.

cURL and libcurl

curl on z/os, symbian and os21

The curl project continues to conquer new markets and it continues to get ported to more platforms and operating systems. Just recently there have been reports about…

  1. A port to the IBM z/OS system, with the official IBM info about it being found here.
  2. Dan Fandrich ported it to the Symbian OS
  3. Christian Vogt mentioned that he had no troubles porting and using it on OS21

I’m trying to maintain a list of all CPUs and operating systems we have known ports being run on and if you have curl and/or libcurl running on another than the ones I list at the bottom of the INSTALL document, please let me know!

cURL and libcurl, Rockbox

More Means Less

1 Comment

Less is more it is said, and I can certainly subscribe to the reverse: more means less. The two primary open source projects I spend time in have been growing the last years, in source code contributions, but also in amount of users and in amount of contributors. I see the similar effects on myself and on my own role in both Rockbox and curl: I do more and more coordination, planning, admin work, talk (chatting on IRC, responding to mails etc) and “guidance” than actual coding work. My code/non-code work ratio has decreased massively.

This is not complaint, just an observation!

It makes sense to me that early on in a project, and until there’s enough momentum to get the project to more or less drive itself, it is important with a driving core that pushes the project forward. That makes sure every little peace fits together and gets the proper attention to make it a good product and project. As time goes by, more and more people get that knowledge, that ability and the amount of people that drive the project forward increases.

So being an “elderly” in both these projects, I’m more of an advisor, talker, tinker, admin, than a lead programmer now. This is at least most notable in Rockbox, since we have 80 committers now and I think at least 50 of them are active.

I probably spend roughly the same amount of time: somewhere around 2-3 hours/day on my open source projects.

Of course, in my particular case exactly now, I’ve also just recently ramped up my working hours and find myself trying to get accustomed to this life with full-time work, a two-kids-and-wife family and several time-consuming spare time projects. It takes a great deal of juggling and less sleeping.

Nothing is forever so I’m certain my situation will change over time. I’m determined to continue hacking in both projects. And my juggling skills will improve…

cURL and libcurl

Why curl sticks with CVS

Occasionally people ask me or just mock me because we’re still using CVS in the curl project, even though there are much more compelling alternatives out there now. Subversion, git, Bzr, Mercurial, etc. I am even a contributor and committer in the Subversion project. (Although I’ll be the first to admit that I never committed much and the stuff I did was done many years ago.)

CVS just isn’t bad enough to warrant the work of a replacement. curl is a tiny project (source code wise) and while CVS has several flaws in how it is designed and works, those flaws never hurt us much. Basically the only one is the lack of rename support and that has no major impact on us.

On the contrary, CVS has the upside of being established and rock solid since many years so people on all sorts of platforms can use it and get the curl source code. This is important especially for our automated build-system which we try hard to find volunteers for to run automatically daily around-the-clock (the results and outputs are then mailed to our central autobuild master server that collects and presents them) and then those guys need to be able to checkout the code easily. Using more modern tools will make it harder since those aren’t available as widely as binary packages for as many (outdated) platforms as CVS is.

So curl sticks with CVS for now.

cURL and libcurl

No metalink in libcurl

1 Comment

It’s been a while since we had this discussion so I figure it is about time to re-iterate it and this time I thought I’d do a little blog post to put the lights on my stand-point regarding this issue:

metalink support in libcurl

I’ve had this discussion at length with Anthony Bryan (the main man behind the metalink format) privately in the past and I’ve bounced back a lot of feedback on the actual XML format to him and I believe some of that were taken into account and changed the format. Of course this was before it “settled” and started to get adopted. I think metalink is a great idea and the file format is (the last time I checked it out, I can’t seem to find the docs now) mostly making sense.

libcurlI have little to no understanding for the idea that libcurl should add support for this natively. metalink is just an XML format that sets up resources for an application to where and how it can download files, and libcurl does indeed support most of the protocols that such URLs can use. libcurl is a data transfer library that is oriented around a given URL and the URL in question has a 1:1 relationship to what protocol it is and it is always content-agnostic.

metalink is application layer, not transport. Adding metalink to libcurl would mean that all of a sudden libcurl would transfer a file and actually parse the (XML!) contents of that file and then get (possibly) multiple streams using multiple protocols based on what that parsing gave. It is just so many new things and violations against key libcurl concepts that I cannot see this done.

Metalink isn’t even a standard so we would then more or less open the gates for further random efforts to introduce similar ideas and whatnot and where would we draw the line? Currently I think we have a pretty solid border drawn in the sand and we don’t cross that line (on purpose).

And frankly, there is only one and one reason only (mentioned and that I can think of) for libcurl to support this feature and I that is because libcurl is already widely adopted it would be easier for metalink to conquer the world by sneaking in the back-door with libcurl as then a large amount of applications would support it with no additional efforts at all. But sorry, I don’t think that’s a good enough reason to break or change these important key concepts/limits of libcurl. (Actually, I think it is a bit foolish to think that adding metalink to libcurl would make all these applications automatically support metalink as there would be several arguments against that too.)

As I’ve said before, I think one of our biggest challenges in this project is to limit what libcurl does, to not allow it to grow in all directions, to keep the scope and to maintain focus.

A metalink file transfer library could be made as a layer on top of libcurl, and I think that is the only logical and sensible way.

Adding metalink support to the curl tool however, seems like a good idea to me…

cURL and libcurl

curl and libcurl 7.18.1

Mainly thanks to the 22 friends named in the release notes, curl and libcurl 7.18.1 was released today with the news and fixes that should prove this the best curl and libcurl versions ever – I guess we always have to believe that our latest is the greatest, why else would we release it?

cURL

The release notes identifies 23 bug fixes we did during the two months since the last release, and the news we introduce include these goodies:

  • added support for “HttpOnly” cookies
  • ‘make ca-bundle’ downloads and generates an updated ca bundle file
  • we no longer distribute or install a ca cert bundle
  • SSLv2 is now disabled by default for SSL operations
  • the test509-style setting URL in callback is officially no longer supported
  • support a full chain of certificates in a given PKCS12 certificate
  • resumed transfers work with SFTP
  • added type checking macros for curl_easy_setopt() and curl_easy_getinfo(), watch out for new warnings in code using libcurl (needs gcc-4.3 and currently only works in C mode)
  • curl_easy_setopt(), curl_easy_getinfo(), curl_share_setopt() and curl_multi_setopt() uses are now checked to use exactly three arguments
  • –with-ca-path=DIR configure option allows to set an openSSL CApath instead of a default ca bundle.

As usual, you can download it here.

cURL and libcurl, Rockbox

Two for the price of one

As probably one of the last humans on earth, I got my paper version of Linux Journal’s April 2008 issue in my mailbox today. While I’ve felt that the magazine has slowly degraded over the last years, this issue certainly made me all warm and fuzzy inside since it contains not one but two articles on subjects very dear to me. A three-page article was dedicated to a Rockbox review (subscription-required-to-read link) named “iPod + Rockbox =Linux Journal April 2008 issue 168 Entertainment Extravaganza” and there was a two-page article on how to use curl (another paid-subscription-required link)to fiddle with a web interface named “Twittering from the Command Line”.

I have a hard time to put a finger on exactly why I feel that the magazine has lost some of its past qualities but I’ve noticed that I finish the magazine much faster these days and I generally skip over more uninteresting articles. I haven’t really investigated this, but it may very well be because a shift in focus in the magazine. I tend to be interested in low-level, kernel, OS, hardware-touching, libraries and related stuff. Not so much in high-level applications or web frameworks and more user and desktop oriented fluff.

I should also add that the Rockbox review was pretty good. Not perfect in any way since it contained errors, was for an old boring grayscale-ipod target and suggested that the original OS is better suited for playing music… The article that mentioned curl wasn’t really about curl but only showed how it can be used but it was mentioned in positive terms at least!