All posts by Daniel Stenberg

curl 8.11.1

Welcome to another curl release. This time we do a bugfix only release, five weeks since the previous version shipped.

Release Presentation

Numbers

the 263rd release
0 changes
35 days (total: 9,763)

79 bugfixes (total: 11,173)
115 commits (total: 33,811)
0 new public libcurl function (total: 94)
0 new curl_easy_setopt() option (total: 306)

0 new curl command line option (total: 266)
51 contributors, 32 new (total: 3,299)
22 authors, 10 new (total: 1,323)
1 security fixes (total: 161)

Security

CVE-2024-11053: netrc and redirect credential leak. (Severity: Low) When asked to both use a .netrc file for credentials and to follow HTTP redirects, curl could leak the password used for the first host to the followed-to host under certain circumstances.

Bugfixes

As usual, here follows some bugfixes I figure could be worth highlighting. See the changelog on the curl site for the full list of changes.

curl

  • –continue-at is mutually exclusive with –no-clobber
  • –continue-at is mutually exclusive with –range
  • –continue-at is mutually exclusive with –remove-on-error
  • use real time in trace timestamps

scripts

  • dmaketgz: use –no-cache when building docker image

libcurl

  • duphandle: also init netrc
  • hostip: don’t use the resolver for FQDN localhost
  • mime: fix reader stall on small read lengths
  • mprintf: fix integer overflow checks
  • multi: fix callback for CURLMOPT_TIMERFUNCTION not being called again
  • netrc: address several netrc parser flaws
  • netrc: support large file, longer lines, longer tokens
  • socket: handle binding to “host!”

http related

  • http_negotiate: allow for a one byte larger channel binding buffer
  • digest: produce a shorter cnonce in Digest headers
  • cookie: treat cookie name case sensitively
  • nghttp2: use custom memory functions

protocols

  • libssh: use libssh sftp_aio to upload file
  • libssh: when using IPv6 numerical address, add brackets
  • OpenSSL: improved error message on expired certificate
  • rtsp: check EOS in the RTSP receive and return an error code
  • schannel: remove TLS 1.3 ciphersuite-list support
  • fixes for wolfSSL OPENSSL_COEXIST

No need to email me about Cisco AnyConnect

My name and email address can be found in the VPN client application made by Cisco called AnyConnect.

They are present there as part of the curl license, because this product – like thousands of others – uses libcurl. My name appears in many products.

Apparently, people often have problems finding an appropriate address to contact when they have issues with this app. This leads a disproportionate amount of them to send emails to me asking for solutions and fixes to their situations.

So far over the years, close to one hundred different persons have emailed me about problems with Cisco’s AnyConnect. I have not been able to help a single one of them because I know nothing about this application.

The reason my email address is shown there is because I am the lead developer of curl, which is but a small component in this application. I am not associated with Cisco nor this product.

This is the support email address you are looking for:

ac-mobile-feedback@cisco.com

See also: other funny emails I got and curl credit screenshots.

Workshop season six, episode three

One positive thing among many others at this version of the HTTP Workshop (day one, day two) is the fact that there have been several new faces showing up here. People who have not previously attended any HTTP Workshops. Getting fresh blood into the mix is great. A chance to maybe lower the average age of the attendees also feels welcome.

This half day was the final session for this time. Three topics were dealt with.

Do you speak HTTP? Getting your HTTP implementation to do right according to the specification can be a challenge. There is a whole range of existing tests for various areas of HTTP but there might still be a place to add HTTP semantic tests in particular for servers. Discussions brought about reflections around testing, doing tests, test formats, other tests, test infrastructure and more. I think the general sense was that yes it would be great. At least if someone else makes it happen…

Every HTTP stack is an intermediary – HTTP semantics is the (requirement for low-level) API. Yes. Lots of nodding around the huge table.

Workshop feedback and thoughts. What is a good cadence for future events, how long should the events be etc. This is probably the maximum amount of attendees we can handle using the same setup. This event was clearly better than several of the past ones in terms of diversity, but I will second our “workshop maestro” in that it could improve further still. We also discussed whether do-arranging together with IETF is good or bad, should it then be before or after IETF?

I think the consensus said that making it biannual event is good. The reasoning for keeping the event in Europe has been because a larger share of the European attendees come from smaller companies compared to the non-Europeans which to a larger degree come from larger companies that might have it easier to pay for longer trips.

My personal take

The HTTP Workshop is a one-of-a-kind event. At these events everything is about and around HTTP with an information density level that is super high. We get to learn how things actually work for people or that do not work. And that we are not alone in whatever struggles or HTTP challenges we have.

Networking with other doers here and absorbing every protocol detail being expressed, gives food for thoughts and lessons to take advantage from in years to come when we for sure are going to take HTTP transfers further. This is in many ways a kind of brain fertilizer event.

Did I mention I enjoyed it? I will certainly try to attend the next one.

The 2024 Workshop, day two

The fun continues. See day one.

In an office building close to the Waterloo station in London, around 40 persons again sat down at this giant table forming a big square that made it possible for us all to see each other. One by one there were brief presentations done with follow-up discussions. The discussions often reiterated old truths, brought up related topics and sometimes went deep down into the weeds about teeny weeny details of the involved protocol specs. The way we love it.

The people around the table represent Ericsson, Google, Microsoft, Apple, Meta, Akamai, Cloudflare, Fastly, Mozilla, Varnish. Caddy, Nginx, Haproxy, Tomcat, Adobe and curl and probably a few more I forget now. One could say with some level of certainty that a large portion of every day HTTP traffic in the world is managed by things managed by people present here.

This morning we all actually understood that the south entrance is actually the east one (yeah, that’s a so called internal joke) and most of us were sitting down, eager and prepared when the day started at 9:30 am.

Capsule. The capsule protocol (RFC 9297) is a way to, put simply, send UDP packets/datagrams over old style HTTP/1 or HTTP/2 proxies.

Cookies. With the 6265bis effort well on its way to ship as an updated RFC, there is an effort and intent to take yet another stab at improving and refreshing the cookie spec situation. In particular to better split off browser management and API related stuff from the more network-oriented over-the-wire details. You know yours truly never ceases an opportunity to voice his opinion on cookies… I approve of this attempt as well, as I think increasing clarity and improving the specification situation can’t but to help improve things.

Declarative web push. There’s an ongoing effort to improve web push – not to be confused with server push, so that it can be done easier and without needing JavaScript to manage it in the client side.

Reverse HTTP. There are origins who want to contact their CDNs without having to listening on any ports/sockets and still be able to provide content. That’s one of the use cases for Reverse HTTP and we got to learn details from internet drafts done on the topics for the last fifteen years and why it might still be a worthwhile effort and why the use cases still exist. But is there an enough demand to put it into HTTP?

Server Stack Detection. A discussion around how someone can detect the origin of the server stack of any given HTTP server implementation. Should there be a better way? What is the downside of introducing what would basically be the server version of the user-agent header field? Lots of productive discussions on how to avoid recreating problems of the past but in a reversed way.

MoQ: What is it and why is it not just HTTP/3? Was an educational session about the ongoing work done in this working group that is wrongly named and would appreciate more input from the general protocol community.

New HTTP stack. A description of the journey of a full HTTP stack rewrite: how components can be chained together and in which order and a follow-up discussion about if this should be included in documentation and if so in which way etc. Lessons included that the spec is one thing, the Internet is another. Maybe not an entirely new revelation.

Multiplexing in the year 2024. There are details in HTTP/2 multiplexing that does not really work, there are assumptions that are now hard to change. To introduce new protocols and features in the modern HTTP stack, things need to be done for both HTTP/2 and HTTP/3 that are similar but still different and it forces additional work and pain.

What if we create a way to do multiplexing over TCP, so called “over streams”, so that the HTTP/3 fallback over TCP could still be done using HTTP/3 framing. This would allow future new protocols to remain HTTP/3-only and just make the transport be either QUIC+UDP or Streams-over TCP+TLS. This triggered a lot of discussions, mostly positive and forward-looking but also a lot of concerns raised about additional work and yet another protocols component to write and implement that then needs to be supported until the end of times because things never truly go away completely.

I think this sounds like a fun challenge! Count me in.

End of day two. I need a beer or two to digest this.

The 2024 HTTP Workshop

Day one.

For the sixth time, this informal group of HTTP implementers and related “interested parties” unite in a room over a couple of days doing a HTTP Workshop. Nine years since that first event in Münster, Germany.

If you are someone like me, obsessed with networking and HTTP in particular this is certainly the place to be. Talking and discussing past lessons, coming changes and protocol dreams in several days with like-minded people is a blast. The people on these events are friends that I don’t always get to hang out with too often. Many of the attendees here have been involved in this community for a long time and have attended all or most of the past workshops as well. I have fortunately been able to attend all of them so far.

This time we are in London.

Let me tell you a little about the topics of day one – without spilling the beans about exactly who said what or what the company they came from. (I will add links to the presentations later once I know where to link to.)

Make Cleartext HTTP harder. The first discussion point of the day. While we have made HTTPS easier over recent years it can only take us so far. What if we considered means to make HTTP harder to use as the next level efforts to further reduce its use over the internet. The HSTS preload list is only growing. Should we instead convert it into a HSTS exclude list that can shrink over time? This triggered a looong a discussion in the group which brought back a lot of old arguments and reasoning from days I thought we had left behind long ago.

HTTP 2/3 abuses. A prominent implementer of a HTTP proxy/load balancer walked us through a whole series of different HTTP/2 and HTTP/3 protocols details that attackers can have been exploiting in recent years, with details about what can be done to mitigate such attacks. It made several other implementers mention how they take similar precautions and some other general discussions around the topics of what can be considered normal use of the protocol and what is not.

Idle connections & mobile: beneficial or harmful. 0-RTT instead of idle connections. There is a non-negligible cost associate with keeping connections alive for clients running on mobile phones. Would it be possible to instead move forward into a world when they are not kept alive but instead closed and 0-RTT opened again next time they are needed? Again the room woke up to a long discussion about the benefits and problems with doing this – which if it would be possible probably would save a lot of battery time on the average mobile phones.

QUIC pacing. We learned that it is very important for servers to implement decent QUIC pacing as it can increase performance up to 20 times compared to no pacing at all. What about using the flow control properly? What about changing the default buffer sizes for UDP sockets in the Linux kernel to something similar to TCP sockets in order to help the default case to perform better?

HTTP prioritization for product performance. The HTTP/2 way of doing prioritization was deemed a failure and too complicated a long time ago but in this presentation we were taught that there are definitely use cases and scenarios where the regular HTTP/3 priority setup is helpful and improves performs. Examples and descriptions for a popular and well used client were shown.

Allowing HTTP clients to use stale DNS data. What if HTTP clients would use stale data instead of having to wait for the DNS resolver response as a means to avoid having to wait a whole RTT to get the date that in a fair amount of the cases is the same as the stale data. Again a long discussion around TTLs for DNS queries and the fact that some clients are already doing this, in more or less explicit ways.

QUIC cache DSR. As the last talk of the the afternoon we got in the details of Direct Server Response for QUIC and how this can improve performance and problems and challenges involved with this. It then indirectly took us into a long sub-thread talking about HTTP caching, Vary headers and what could and should be done to improve things going forward. There seems to be an understanding that it would be good to improve the current situation but it is not entirely clear to this author exactly what that would entail.

When we then took a walk through the streets of London, only to have an awesome dinner during which we could all conclude that HTTP still is not ready. There is still work to be done. There are challenges left to overcome.

See you tomorrow for day two.

Rock-solid curl

I am thrilled to announce:

Rock-Solid curl: long term supported curl releases

Basics

We make long term support releases of curl that we call Rock-solid curl.

We support each release branch for at least five years.

We only merge security fixes and important stability bugfixes into these branches for updates. No new features. No surprises.

We offer Rock-solid curl downloads to existing support customers. It means that there is no free and open access to these releases. To get access, become a customer!

Rock-solid curl is released under the same license as normal curl (or optionally a commercial license). No funny business.

Rock-solid curl is meant to greatly reduce the risk of regressions and yet be a safe and secure solution with full support. For the companies who want this extra level of attention. An even smoother ride.

We plan to make new Rock-solid curl release branches roughly every 18-24 months.

The first Rock-solid curl release

Rock-solid curl 8.9.2 is the first long-term support curl version. As the version number implies, it is based on the curl 8.9.1 release that we shipped back in July, with two security fixes and a small number of stability patches applied.

Once you have a contract with us, you can get it.

Who is doing this?

I, Daniel Stenberg, will be the primary support person for Rock-solid curl and I will do the releases, and most of the patching and the back-porting of what is deemed necessary.

Customers sign contracts with wolfSSL for this. wolfSSL pays my salary. I have worked for and with wolfSSL with this business setup since 2019.

What about “the normal” curl?

Nothing changes with or happens to the curl project and the regular curl releases because of this. No one is going anywhere. The curl license remains the same. The curl releases and the release cadence remain intact.

Support customers help fund the project by allowing us to pay developers.

How do I become a customer?

Head over to rock-solid.curl.dev and contact us via the provided links.

Downloads and all Rock-solid curl information is hosted on the dedicated rock-solid.curl.dev site, separate from the open source project on curl.se.

On curl

Born in the late 1990s, curl is a client-side Internet transfer engine. Installed in over twenty billion instances it serves virtually everything that is internet connected: phones, tablets, cars, television sets, printers, medical devices, game consoles, helicopters on other planets, etc and it is an embedded component in a significant share of our most used and beloved apps, tools, games and services.

curl is the fruit and outcome from hard work by thousands of volunteers and is completely free and Open Source. The curl project is independent. It is not part of any umbrella organization or foundation and it is not owned nor controlled by any company.

curl is secure, fast and feature-rich. It is a defacto standard and key infrastructure.

curl 8.11.0

Numbers

the 262nd release
5 changes
49 days (total: 9,728)

266 bugfixes (total: 11,094)
435 commits (total: 33,694)
0 new public libcurl function (total: 94)
0 new curl_easy_setopt() option (total: 306)

1 new curl command line option (total: 266)
55 contributors, 22 new (total: 3,268)
25 authors, 10 new (total: 1,312)
1 security fixes (total: 160)

Release presentation

Security

CVE-2024-9681: HSTS subdomain overwrites parent cache entry. When curl is asked to use HSTS, the expiry time for a subdomain might overwrite a parent domain’s cache entry, making it end sooner or later than otherwise intended.

Changes

  • –create-dirs works for –dump-header as well
  • P12 format support added to GnuTLS backend
  • Added options to disable IPFS
  • TLSv1.3 earlydata support (with GnuTLS)
  • Official WebSocket support

Bugfixes

These are some of my favorite bugfixes in this release.

Build

  • cmake: document -D and env build options
  • configure: add support for ‘unity’ builds
  • configure: set linker flags to allow rustls build on macos

curl

  • detect ECH support dynamically, not at build time
  • support –show-headers AND –remote-header-name
  • make –skip-existing work for –parallel

libcurl

  • conncache: find bundle again in case it is removed
  • curl.h: remove the struct pointer for CURL/CURLSH/CURLM typedefs
  • ftp: fix 0-length last write on upload from stdin
  • hsts: support “implied LWS” properly around max-age
  • lib: remove function pointer typecasts for hmac/sha256/md5
  • mprintf: do not ignore length modifiers of %o, %x, %X
  • mprintf: treat %o as unsigned
  • multi: make curl_multi_cleanup invalidate magic latter
  • multi: make multi_handle_timeout use the connect timeout
  • netrc: cache the netrc file in memory
  • select: use poll() if existing, avoid poll() with no sockets
  • url: use same credentials on redirect
  • urlapi: normalize the IPv6 address

protocols

  • ngtcp2: set max window size to 10x of initial (128KB)
  • url: connection reuse on h3 connections
  • gnutls: use session cache for QUIC
  • mbedTLS: fix handling of TLSv1.3 sessions
  • schannel: ignore error on recv beyond close notify
  • schannel: reclassify extra-verbose schannel_recv messages
  • quic: use send/recvmmsg when available
  • quic: use the session cache with wolfSSL as well

tests

  • generate lib1521.c atomically
  • remove all valgrind disable instructions
  • remove debug requirement on 38 tests
  • use ‘-4’ where needed

Next

Unless we find a terrible regression, the next curl release is scheduled to ship on January 8, 2025.

curl source code age

In every software project that has been around for a while there is of course newer code and older code. A question that often pops up at least in my mind is then: How much of the old code has actually survived over the years and is still being in use today?

And how would you visualize that in a way that makes it possible to understand the data?

A challenge

This turned out to become my challenge of the week.

I started off writing a script that iterates over all release tags we have set in the curl git repository and for every such tag, it extracts all relevant source files and runs git blame on them. With the -t --line-porcelain options, the output is really easy to parse.

For every such release tag, we get a large number of lines with different timestamps. Then the script sorts all those timestamps, iterates over them, counts how many that were done within different intervals in time and outputs those counters in a formatted line.

Iterating over several hundred tags in a code base of curl size and running git blame like this is not a quick operation. On my decently fast machine, a full such round takes well over an hour. Admittedly there is probably ways the algorithm can be improved.

gnuplot

Once all the data is written, it is converted into a visualization using gnuplot. I needed to experiment. I had to experiment a bit to learn how to do filledcurves.

Take 1

My first take split up the age just as a percentage. How large share of the code has been changed within how many months.

Turned out rather hard to interpret and understand.

Take 2

The source code is always 100%, so how large share of the source code is written within which two-year segment?

I decided to split the time in two-year segments only to keep the number of segments down a little.

Also, I moved the labels to the right side as it is the side where you are most likely interested in reading them. I had to put the legend outside of the graph.

While I think this version turned out pretty cool, the actual number of lines of code and the growth of the code base is completely invisible in this version.

Take 3

What if I would do the take 2 version but do it based on actual number lines instead. I poked the script again, restarted it and let it run for another hour or two.

Better! This version shows the segments in a way that properly reflects the actual number of lines over time. It beats the weird percentage take from above.

Still, having the oldest code slide over on top of the graph like this and have newer code appear from below might not be the best way to illustrate this data. What if I instead swapped it around so that the graph would keep the oldest code at the bottom and add newer code over that?

Take 4

I think this shows perfectly fine how the exact same data can be experienced so much better if shown just slightly differently.

In this version below, I also experimented a bit on how to name the segments in the legend as someone pointed out that it may not be entirely obvious to everyone that I do two-year segments.

I could also move the legend into the graph again here.

Pedantic viewers of this graph will spot how the number of lines of code here is slightly different than the separate line of code graph shown in the curl dashboard. This, because git blame includes all the lines and the other graph is done using cloc which excludes blank lines – and probably some other minor differences as well.

Take 4 is the version of the scripts that starting now will be included in the curl dashboard.

Takeaways

More than 50% of existing curl code was written since 2020.

About 25% of existing code was written before 2014.

Almost 10% was written before 2008.

1254 lines (0.64%) are still left in the code that were written before the year 2000.

No, I don’t know how this compares to other projects of similar age.

The scripts

codeage.pl and codeage.plot

If you want to play with them against your own git repositories, you will notice that there are some curl-specific assumptions in there that you need to address, but that should not be difficult to patch.

Follow-up

Kees Cook wrote an adaptation of these scripts to generate the same graph for the Linux kernel. In Python and using multiple threads.