Monthly Archives: July 2024

cURL and libcurl

curl 8.9.1

Some annoying regressions triggered this.

Numbers

the 259th release
0 changes
7 days (total: 9,630)
28 bugfixes (total: 10,559)
43 commits (total: 32,748)
0 new public libcurl function (total: 94)
0 new curl_easy_setopt() option (total: 306)
0 new curl command line option (total: 263)
19 contributors, 5 new (total: 3,211)
10 authors, 1 new (total: 1,288)
1 security fixes (total: 158)

Download the new curl release from curl.se as always.

Release presentation

Security

We decided to do a patch release. Then yesterday we got a security vulnerability reported and so now we have that fixed in here as well.

CVE-2024-7264: ASN.1 date parser overread (severity low) libcurl’s ASN1 parser code has the GTime2str() function, used for parsing an ASN.1 Generalized Time field. If given an syntactically incorrect field, the parser might end up using -1 for the length of the time fraction, leading to a strlen() getting performed on a pointer to a heap buffer area that is not (purposely) null terminated.

Bugfixes

This release is done only because we shipped a few regressions in 8.9.0 we rather let users avoid. Here are some noteworthy fixes from the past week:

  • connection shutdown fix for event based processing – this would cause applications to keep monitoring sockets “too much”, easily leading to busy-loops or worse
  • cmake builds detect libssh and nettle better
  • several libcurl functions now survive NULL pointer inputs better
  • fixed an Apple SDK bug workaround for non-macOS targets
  • the curl tool builds with the manual enabled on OS400
  • works around an IBM (OS400) ASCII run-time library bug
  • speed limiting for 32bit systems had the wrong math
  • allow wolfSSL’s implementation of kyber to be used
  • wolfssl CA store caching fix
  • more defensive and portable socket code for the curl tool’s --ip-tos logic
cURL and libcurl

changelog changes

On the curl website we of course list exactly what changes that go into each and every single release we do. In recent years I have even gone back and made sure we provide this information for every single release ever done. At the moment that means 258 releases, listing over 10,000 bugfixes and almost 1,000 changes. From 1996 until today.

This is literally a wall of changes.

Since we keep doing somewhere between 150 and 250 bugfixes per release and we do a new release very eight weeks, the page with all changes these keeps growing quite fast.

Right now, the HTML of this page is at 1.1 megabytes.

Use case

Most typically I think the use case for users visiting the changelog is to view what changes that were done in one specific curl release. Possibly checking out a few different ones. Very few users actually want tens of thousands of lines of text to scroll through. I believe.

Enter single release changelogs

To make sure that people can read the changes for a single release only, and to reduce the amount of data a user needs to download in order to view those single release changes, I worked on a setup that generates separate individual changelog pages for every release. Easy to bookmark, load fast, contain only information about the specific releases and they make it easy to skip back and forth between past and future releases.

I deployed these changes today and if you go to https://curl.se/ch/ now, you will see the changelog for the most recent release only.

The all changes changelog remains

The changelog showing everything will remain and is still an option to browse. I personally use it at times when I want to control-f and look for a change done in a previous curl version that I cannot remember exactly which. This all-changes page remains only a click away if you rather view that one instead of the single-version thing.

Design

I am not a web developer and I am not web designer. I know just enough HTML and CSS to be able to publish these things, but I do not do fancy and I am fully aware that I am not good at making “nice” or “attractive” designs. I focus on usable and practical.

As per curl website standards these pages are all static content using no JavaScript and only a few small images. Excellent for rendering fast and for caching well in the CDN.

Known vulnerabilities

I did not especially mention this before, but only a few days ago I added direct links from each version header to the page for known vulnerabilities for that specific version and that link of course is now also present in the single version changelog page. Next to the link that goes directly to the release presentation video.

Screenshot of the changelog page for curl 8.8.0

Feedback?

If you find problems or have ideas on how to further improve the curl website, let us know!

cURL and libcurl

curl 8.9.0

Numbers

the 258th release
11 changes
63 days (total: 9,623)
260 bugfixes (total: 10,531)
423 commits (total: 32,704)
0 new public libcurl function (total: 94)
1 new curl_easy_setopt() option (total: 306)
4 new curl command line option (total: 263)
80 contributors, 38 new (total: 3,209)
47 authors, 16 new (total: 1,288)
2 security fixes (total: 157)

Download the new curl release from curl.se as always.

Release presentation

Security

Today we fix two security vulnerabilities and publish all details about them.

  • CVE-2024-6197: freeing stack buffer in utf8asn1str. (severity medium) libcurl’s ASN1 parser has this utf8asn1str() function used for parsing an ASN.1 UTF-8 string. It can detect an invalid field and return error. Unfortunately, when doing so it also invokes free() on a 4 byte local stack buffer.
  • CVE-2024-6874: macidn punycode buffer overread. (severity low) libcurl’s URL API function curl_url_get() offers punycode conversions, to and from IDN. Asking to convert a name that is exactly 256 bytes, libcurl ends up reading outside of a stack based buffer when built to use the macidn IDN backend. The conversion function then fills up the provided buffer exactly – but does not null terminate the string.

Changes

  • –ip-tos (IP Type of Service / Traffic Class). Lets users set this IP header field to a number.
  • –mptcp. Asks curl to enable the Multipath TCP option for this connection, which if the server also allows it may make the TCP connection to go over multiple network paths.
  • –vlan-priority. Makes curl set the VLAN priority field for its IP traffic. This is typically a field used in the network layer below IP (think Ethernet), so it is not likely to survive through IP routers. A local network thing.
  • –keepalive-cnt (and CURLOPT_TCP_KEEPCNT). Specify how many keeplive probes curl should send before it considers the connection to be dead.
  • –write-out ‘%{num_retries}’ is a new variable for the info output that outputs the number of retries that were done for the previous transfer (when –retry was used).
  • gnutls now supports CA caching. For libcurl using applications, this can really speed up doing serial TLS connections.
  • mbedtls supports CURLOPT_CERTINFO. Returns certificate information to the application.
  • noproxy patterns need to be comma separated. Space separation is no longer enough.
  • Support binding a connection to both interface and IP, not just one of them.
  • The URL API added CURLU_NO_GUESS_SCHEME, to allow an application to figure out if the scheme for a previously parsed URL was set or guessed.
  • wolfssl now supports CA caching

Bugfixes

In no other release ever before in curl’s long history have there been this many bugfixes: 260. Some of my favorites are:

  • cmake: 26 separate bugfixes
  • configure: 10 separate bugfixes
  • –help category cleanup and list categories in –help
  • allow etag and content-disposition for 3xx reply
  • docs: countless fixes, polish and corections
  • show name and keywords for failed tests in summary
  • avoid using GetAddrInfoExW with impersonation
  • URL encode the canonical path for aws-sigv4
  • fix DoH cleanup
  • fix memory leak and zero-length HTTPS RR crash in DoH
  • allow DoH transfers to override max connection limit
  • fix ß with AppleIDN
  • fix compilation with OpenSSL 1.x with md4 disabled
  • do a final progress update on connect failure
  • multi: fix pollset during RESOLVING phase
  • enable UDP GRO for QUIC
  • require at least OpenSSL 3.3 for QUIC
  • add shutdown support for HTTP/3 (QUIC)
  • fix CRLF conversion of input
  • fixed starttls for SMTP
  • change TCP keepalive from ms to seconds on DragonFly BSD
  • support TCP keepalive parameters on Solaris <11.4
  • shutdown TLS and TCP better
  • gnutls: pass in SNI name, not hostname when checking cert
  • gnutls: rectify the TLS version checks for QUIC
  • mbedtls v3.6.0 workarounds
  • several x509 asn.1 parser fixes

Next

Because the 8.9.0 release spent an extra week for its release cycle, the next one is going to be one week shorter. We do this by shortening the feature window to just two weeks this time, which might impact how many new features and changes we manage to merge.

We have a large amount of pull requests for changes already pending merge, waiting for the release window to open.

If all goes well, the next release is named 8.10.0 and eventually ships on September 11, 2024.

cURL and libcurl, wolfSSL

curl for QNX

Starting now, there are official curl releases for QNX hosted on the curl.se website. See https://curl.se/qnx.

QNX is a commercial real-time operating system and these curl release packages are produced as a result of a business arrangement.

The plan is to from now on ship curl tarballs for three different QNX versions, and each archive contains curl and libcurl built for several different targets. The curl for QNX releases should be possible to release in sync with the regular releases, but they can also be updated out of sync if need be.

Every curl release from here on out will be packaged for QNX and made available.

curl and libcurl have been functional on QNX since decades – the first mention of curl and QNX together that I could find is from October 2000. curl releases for QNX were previously packaged and provided to end users by the QNX team themselves.

This move will allow QNX users to get the latest curl faster and make them able to keep up better with curl development. For features, bugfixes and perhaps most of all security.

We will also make sure that curl keeps building fine for QNX straight from the tarball.

The complete set of build and setup scripts for curl on QNX are maintained in the curl-for-qnx git repository. Of course we will appreciate submitted issues and pull requests in that repository as well.

This commercial agreement is between Blackberry and wolfSSL. I am employed by wolfSSL. If you want your operating system to have equally fancy and always up-to-date releases, you know who to contact.

cURL and libcurl

wcurl is here

4 Comments

Users tell us that remembering what curl options to use when they just want to download the contents of a URL is hard. This is one often repeated reason why some users reach for wget instead of curl on the command line. It downloads the data from the URL without you needing to provide any extra arguments. Without you needing to remember which option(s) to use.

In the curl user survey of 2024, it was again mentioned several times.

Enter wcurl

Samuel Henrique decided to do something about it. Today he announced that he not only created wcurl as a curl wrapper aimed at meeting this exact need, he also created a Debian package out of it and made sure wcurl now ships as part of the curl package. Starting in 8.8.0-2. I already have it on my Debian unstable installations.

wcurl is implemented a shell script that uses curl. It also ships with its own manpage.

Take it for a spin. Tell the team what you think!

Discussion

Hacker news