Since Henrik replied to my previous blog posts, I figured I better write a new one to simply state the fact:
…is now up and working. Go there and read all about it! And yeah, my curl talk is currently set for 15:00 on that Saturday.
Category Archives: cURL and libcurl
curl and/or libcurl related
curl vs wget, really
Ok, since people truly and actually often ask me about what the differences are between curl and Wget, it might be suitable to throw in my garbage here and state the main differences as I see them. Please consider my bias towards curl since after all, curl is my baby – but I have contributed code to Wget as well.
curl
- Features and is powered by libcurl -a cross-platform library with a stable API that can be used by each and everyone. This difference is major since it creates a completely different attitude on how to do things internally. It is also slightly harder to make a library than a “mere” command line tool.
- Pipes. curl is more in the traditional unix-style, it sends more stuff to stdout, and reads more from stdin in a “everything is a pipe” manner.
- Return codes. curl returns a range of defined and documented return codes for various (error) situations.
- Single shot. curl is bascially made to do single-shot transfers of data.It transfers just the URLs that
the user specifies, and does not contain any recursive downloading logic or any sort of HTML parser. - More protocols. curl supports FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE at the time of this writing. Wget supports HTTP, HTTPS and FTP.
- More portable. Ironically curl builds and runs on lots of more platforms than wget, in spite of their attempts to keep things conservative. For example, VMS, OS/400, TPF and other more “exotic” platforms that aren’t straight-forward unix clones.
- More SSL libraries and SSL support. curl can be built with one out of four different SSL/TLS libraries, and it offers more control and wider support for protocol details.
- curl (or rather libcurl) supports more HTTP authentication methods, and especially when you try over HTTP proxies.
wget
- Wget is command line only. There’s no lib or anything. Personally I’ve always disliked the project doesn’t provide a man page as they stand in the GNU side of this and consider “info” pages to be the superior way to document things like this. I strongly disagree.
- Recursive! Wget’s major strong side compared to curl is its ability to download recursively, or even just download everything that is referred to from a remote resource, be it a HTML page or a FTP directory listing.
- Older. Wget has traces back to 1995, while curl can be tracked back no longer than 1997.
- Less developer activity. While this can be debated, I consider three metrics here: mailing list activity, source code commit frequency and release frequency. Anyone following these two projects can see that the curl project has a lot higher pace in all these areas, and it has indeed been so for several years.
- HTTP 1.0. Wget still does its HTTP operations using HTTP 1.0, and while that is still working remarkably fine and hardly ever is troublesome to the end-users, it is still a fact. curl has done HTTP 1.1 since March 2001 (while still offering optional 1.0 requests).
- GPL. Wget is 100% GPL v2, which I believe is going v3 really soon when they release their next release. curl is MIT licensed.
- GNU. Wget is part of the GNU project and all copyrights are assigned to them etc. The curl project is entirely stand-alone and independent with no organization parenting at all.
This turned out to be a long post and it might in fact be usable to save for the future, so I’m also posting this as a more permanent doc on my site on this URL: http://daniel.haxx.se/docs/curl-vs-wget.html. So possible updates will be done there. Do let me know if you have further evident differences or if you disagree with me on details here!
libcurl in less than 50KB
Gary Maxwell enlightened us that his build (of a slightly older libcurl) is way below 50KB on an ARM7 architecture, while Dan Fandrich could squeeze the latest libcurl release to at least below 100KB on x86.
Of course these particular builds are fairly stripped down builds with only HTTP support left, but they are built from unmodified sources. Full-fledged builds with all protocols will of course be significantly larger.
libcurl bindings
Christopher Smith blogged about improving curlpp and not only did Jean-Philippe react immediately, it also showed me how far away I am from these C++ guys and their ideas and views of the world.
Not only I am not even aware of what functors and facets truly are (nor do I really care), but I find it interesting that the choice of them and whether or not one or the other is used or supported is such a religious thing…
You know what? The older I get, the less interested I get in the maze that is OO concepts. I just so have no interest whatsoever in C++ nor Java!
Of course I’m primarily happy they use libcurl and that they keep enhancing the ways people access it. I have work enough on the C API so I never really dive very deep in the various bindings (there are more than 30 these days).
On a slightly related note, there’s a second Lua binding now called Lua-cURL – the other one is actually called luacurl and yeah the names are… not very imaginative and very very very similar to each other.
fscons curl talk, part II
Ok, it seems the pricing for the entire conference (7-8 dec 2007) is going to be:
Student: 60 euro
Private: 120 euro
Company: 240 euro
The blurb for my particular talk (that is going to be held in English, mind you) that is scheduled for 13.00 on dec 8 is drafted like below:
curl and libcurl is a free software project for data transfers with well known Internet protocols. It’s about to turn 10 years, is downloaded more than 1 million times per year and is the answer to all questions and problems
Daniel Stenberg is the founder and leader of the curl project and will talk about exactly everything that curl is good for, what it does for you, who uses it, who should use it and something about the background, the license, people, development and the culture within the project. Some words about the future and who’s paying for the development will also be dealt with.
And no, the official site still hasn’t gone live so this is still not possible to read elsewhere!
Backwards Compatibility Efforts Gone Broke?
An example from the wild about how hard it can be to satisfy everyone when you’re writing and offering a library to the world: with the recent libcurl release suddenly open office doesn’t link fine with it.
It turns out these guys have enabled our help-define (CURL_NO_OLDIES) always. The define disables all our backwards compatibility defines/fix and let you check that your application still builds with the latest and doesn’t rely on anything that might be removed in the future.
CURL_NO_OLDIES is a convenient define that really suits a purpose. But not really well suited to unconditionally define for all builds since then you of course get these problems when we (in libcurl) rearrange our defines. This problem came to no surprise to us, since we did quite a large rearrangement before this particular release, and I actually expect that these support-defines will be present for a long time ahead.
So not only did they file a Debian bug report on open office, but also on libcurl.
The conclusion: use CURL_NO_OLDIES when you test-build your application against libcurl. Don’t leave it in the Makefile unconditionally for future builds.
curl and libcurl 7.17.0
Just a short while ago I uploaded the 7.17.0 packages to the curl site, updated the front page, mailed the announcement and submitted an update on freshmeat.
The previous release (7.16.4) was done in haste due to the security issue we got reported, but this time we took our time and I think we’ve done a pretty good job. The upcoming weeks will tell for sure…
Changes this time include the new OS/400 port, the fact that curl_easy_setopt() now will copy all strings passed to it (previously it referred to the strings the application still had to keep around), SCP and SFTP support now requires libssh2 0.16 or later, the LDAP libraries are now build-time linked like all other third part libs and no longer run-time dlopen() like before and LDAPS is now supported. We also have at least 27 mentioned bug fixes, and possibly a few more actually done but not detailed in the release notes.
There’s no soname bump this time, although there have been a fair amount of return code name changes (with backwards compatible #defines added to make older programs still possible to compile), but we have started to add stuff in the TODO that we want to do the next time we actually decide to bump again. The previous bump did cause some havoc so I’ve learned to not use that card too often…
Unless something bad creeps up, I’m hoping for a good two months or so until 7.17.1 is due. With my autumn plans it might just take a little while longer as well. Time will have to tell.
Enjoy!
I am Rude and Mean
I’m the maintainer and admin of a few different open source packages, perhaps most notable in the curl project but I also poke on c-ares and libssh2 and I do a fair amount of work on Rockbox and hang around in a few other projects as well.
I write around 400-500 emails a month, the majority of them to the mailing lists of the projects I’m involved in. I try to respond to questions I know the answer to.
I’m not sure if I’ve grown even more grumpy recently or if the world is going downwards, but I’ve recently been called rude and a scammer in public mailing lists after having answered to mails with a meaning that the guys asking the question isn’t exactly trying hard to read up on this, understand the area nor are they reading my answers very good. So what if I’m not always the perfect gentleman or say the right “social” words, I am a hard core tech guy and I answer and talk technical stuff and specific details all day long. That’s what I do and that’s who I am.
So, I just wanted to let the rest of you know: I am rude and mean and you should know better than to ask anything in a forum I frequent. Or then you can of course stand up against the whiners and help educate the world on how to ask questions and why spoon-feeding users on mailing lists isn’t a good idea.
Sticking out your chin in the harsh internet reality, one should expect to get hit like this every now and then and you need to grow pretty thick skin to not let the bad guys get to you. Nonetheless, people in general are nice but it is just too easy for people to get upset and send away very rude mails without any kind of aftermath.
(But I must admit I found the threat to discuss me at a future Zend conference hilarious!)
curl vs wget
ABI Breakage
Domenico Andreoli, the Debian curl and libcurl maintainer posted to the curl-library list about the recent libcurl soname bump and the related issue of ABI breakage.
I responded, explaining my view on why the soname bump from 3 to 4 was valid, but I’ve also gone back in my mail archives and checked out a private discussion I had with Bjorn Reese over this subject a long time ago and I feel it is about time it hits the air.
An ABI breakage occurs when one or more of the following changes are made
Functions:
- Change function names
- Remove existing functions
- Change parameters
- Change behavior
- Change undefined behavior
- Add dependence on other functions (e.g. A() must be called before B())
Data structures:
- Change the order of elements
- Add elements
- Remove elements
- Change the data type or size
- Change alignment
- Change element values (e.g. change bits in flag variables)
Global variables:
- Change variable names or types
- Remove variables
Update: Dan Fandrich posted on the curl-library list and mentioned Mike Hearn’s great related page: Writing shared libraries.