It’s been a while since we had this discussion so I figure it is about time to re-iterate it and this time I thought I’d do a little blog post to put the lights on my stand-point regarding this issue:

metalink support in libcurl

I’ve had this discussion at length with Anthony Bryan (the main man behind the metalink format) privately in the past and I’ve bounced back a lot of feedback on the actual XML format to him and I believe some of that were taken into account and changed the format. Of course this was before it “settled” and started to get adopted. I think metalink is a great idea and the file format is (the last time I checked it out, I can’t seem to find the docs now) mostly making sense.

I have little to no understanding for the idea that libcurl should add support for this natively. metalink is just an XML format that sets up resources for an application to where and how it can download files, and libcurl does indeed support most of the protocols that such URLs can use. libcurl is a data transfer library that is oriented around a given URL and the URL in question has a 1:1 relationship to what protocol it is and it is always content-agnostic.

metalink is application layer, not transport. Adding metalink to libcurl would mean that all of a sudden libcurl would transfer a file and actually parse the (XML!) contents of that file and then get (possibly) multiple streams using multiple protocols based on what that parsing gave. It is just so many new things and violations against key libcurl concepts that I cannot see this done.

Metalink isn’t even a standard so we would then more or less open the gates for further random efforts to introduce similar ideas and whatnot and where would we draw the line? Currently I think we have a pretty solid border drawn in the sand and we don’t cross that line (on purpose).

And frankly, there is only one and one reason only (mentioned and that I can think of) for libcurl to support this feature and I that is because libcurl is already widely adopted it would be easier for metalink to conquer the world by sneaking in the back-door with libcurl as then a large amount of applications would support it with no additional efforts at all. But sorry, I don’t think that’s a good enough reason to break or change these important key concepts/limits of libcurl. (Actually, I think it is a bit foolish to think that adding metalink to libcurl would make all these applications automatically support metalink as there would be several arguments against that too.)

As I’ve said before, I think one of our biggest challenges in this project is to limit what libcurl does, to not allow it to grow in all directions, to keep the scope and to maintain focus.

A metalink file transfer library could be made as a layer on top of libcurl, and I think that is the only logical and sensible way.

Adding metalink support to the curl tool however, seems like a good idea to me…