Welcome to another curl release. This time we do a bugfix only release, five weeks since the previous version shipped.
Release Presentation
Numbers
the 263rd release 0 changes 35 days (total: 9,763) 79 bugfixes (total: 11,173) 115 commits (total: 33,811) 0 new public libcurl function (total: 94) 0 new curl_easy_setopt() option (total: 306) 0 new curl command line option (total: 266) 51 contributors, 32 new (total: 3,299) 22 authors, 10 new (total: 1,323) 1 security fixes (total: 161)
Security
CVE-2024-11053: netrc and redirect credential leak. (Severity: Low) When asked to both use a .netrc file for credentials and to follow HTTP redirects, curl could leak the password used for the first host to the followed-to host under certain circumstances.
Bugfixes
As usual, here follows some bugfixes I figure could be worth highlighting. See the changelog on the curl site for the full list of changes.
curl
–continue-at is mutually exclusive with –no-clobber
–continue-at is mutually exclusive with –range
–continue-at is mutually exclusive with –remove-on-error
use real time in trace timestamps
scripts
dmaketgz: use –no-cache when building docker image
libcurl
duphandle: also init netrc
hostip: don’t use the resolver for FQDN localhost
mime: fix reader stall on small read lengths
mprintf: fix integer overflow checks
multi: fix callback for CURLMOPT_TIMERFUNCTION not being called again
netrc: address several netrc parser flaws
netrc: support large file, longer lines, longer tokens
socket: handle binding to “host!”
http related
http_negotiate: allow for a one byte larger channel binding buffer
digest: produce a shorter cnonce in Digest headers
cookie: treat cookie name case sensitively
nghttp2: use custom memory functions
protocols
libssh: use libssh sftp_aio to upload file
libssh: when using IPv6 numerical address, add brackets
OpenSSL: improved error message on expired certificate
rtsp: check EOS in the RTSP receive and return an error code
Welcome to this follow-up patch release, just a week after we shipped 8.10.0. A bunch of bugfixes.
Numbers
the 261th release 0 changes 7 days (total: 9,679) 24 bugfixes (total: 10,828) 50 commits (total: 33,259) 0 new public libcurl function (total: 94) 0 new curl_easy_setopt() option (total: 306) 0 new curl command line option (total: 265) 19 contributors, 7 new (total: 3,246) 9 authors, 1 new (total: 1,303) 0 security fixes (total: 158)
Download the new curl release from curl.se as always.
Release presentation
Bugfixes
These are the perhaps most important ones fixed this time:
fix configure –with-ca-embed. It could otherwise sometimes lead to an empty bundled CA store.
cmake: ensure CURL_USE_OPENSSL/USE_OPENSSL_QUIC are set in sync
cmake: fix MSH3 to appear on the feature list
runtests: accecpt ‘quictls’ as OpenSSL compatible. It would previously skip a few tests that are marked OpenSSL specific.
connect: store connection info when really done
fix FTP CRLF line endings for ASCII transfer regression. Perhaps most notably this problem was seen on directory listings, which are done using ASCII mode.
fix HTTP/2 end-of-stream handling when uploading data from stdin
http: make max-filesize check not count ignored bodies. Like in the case where a URL is redirected to a second place, the first URL might still provide a body that curl ignores.
fix AF_INET6 use outside of USE_IPV6. Made the build fail on systems without IPv6 support.
check that the multi handle is valid in curl_multi_assign. Perhaps not exactly libcurl’s responsibility, but we found at least one application that did this after the 8.10.0 upgrade.
on QUIC connects, keep on trying on draining server
request: correctly reset the eos_sent flag. When doing multiple HTTP/2 uploads using the same handle – this caused problems for git.
transfer: fix sendrecv() without interim poll. An optimization that optimized a little too much… Most commonly this problem was seen with PHP programs that often (but unwisely) skip the polling.
rustls: fixed minor logic bug in default cipher selection
rustls: support strong CSRNG data. Now every curl build using TLS ensures use of strong random numbers.
trurl is slowing growing up and maturing. This is a minor patch release following up the previous one done just a few weeks ago, fixing a few annoying bugs only.
The query parameter normalization introduced in 0.15 did not properly handle query pairs when one of the sides of the ‘=’ was blank.
Make the generated manpage “source” to use the version number, not the title – which should be plain trurl.
A minuscule escaping mistake in the manual markdown made the output render wrongly.
Only install the manpage for ‘make install’ if there really is a manpage present – since it is generated and bundled in the release tarball it is not necessary present when users build their own
Future
I have this feeling that we still have use cases and combinations that we don’t have tested in the test suite so we probably need to do a few more minor or patch releases until we are ready to bump this baby to 1.0.
the 260th release 18 changes 42 days (total: 9,672) 245 bugfixes (total: 10,804) 461 commits (total: 33,209) 0 new public libcurl function (total: 94) 0 new curl_easy_setopt() option (total: 306) 2 new curl command line option (total: 265) 57 contributors, 28 new (total: 3,239) 27 authors, 14 new (total: 1,302) 1 security fixes (total: 158)
Download the new curl release from curl.se as always.
Release presentation
Security
CVE-2024-8096: OCSP stapling bypass with GnuTLS When curl is told to use the Certificate Status Request TLS extension, often referred to as OCSP stapling, to verify that the server certificate is valid, it might fail to detect some OCSP problems and instead wrongly consider the response as fine.
support for setting TLS version and ciphers for Rustls
stop offering ALPN http/1.1 for http2-prior-knowledge
support for sslcert/sslkey blob options for wolfSSL
release tarball 100% reproducible. We also provide verify-release a convenient shell script allowing anyone and everyone to easily verify curl release tarballs.
Bugfixes
See the full changelog for the complete list. Here follows my favorite subset:
build: add poll() detection for cross-builds
cmake: 40+ bugfixes
configure: fail if PSL is not disabled but not found
runtests: remove “has_textaware”
curl: find curlrc in XDG_CONFIG_HOME without leading dot
curl: make the progress bar detect terminal width changes
curl: bump maximum post data size in memory to 16GB
bearssl/mbedtls/rustls/wolfssl: fix setting tls version
gnutls/wolfssl: improve error message when certificate fails
gnutls: send all data
openssl: certinfo errors now fail correctly
sectransp: fix setting tls version
x509asn1: raise size limit for x509 certification information
ftp: always offer line end conversions
ftp: fix pollset for listening
http2: improved upload eos handling
idn: support non-UTF-8 input under AppleIDN
ngtcp2: use NGHTTP3 prefix instead of NGTCP2 for errors in h3 callbacks
pop3: fix multi-line responses
managen: fix superfluous leading blank line in quoted sections. Nicer HTML version of the manpages.
managen: in man output, remove the leading space from examples
managen: wordwrap long example lines in ASCII output. Nicer curl --manual and -h output.
manpage: ensure a maximum width for the text version.
connect: always prefer ipv6 in IP eyeballing
aws_sigv4: fix canon order for headers with same prefix
cf-socket: prevent KEEPALIVE_FACTOR being set to 1000 for Windows
rand: only provide weak random when needed
sigpipe: init the struct so that first apply ignores
the 259th release 0 changes 7 days (total: 9,630) 28 bugfixes (total: 10,559) 43 commits (total: 32,748) 0 new public libcurl function (total: 94) 0 new curl_easy_setopt() option (total: 306) 0 new curl command line option (total: 263) 19 contributors, 5 new (total: 3,211) 10 authors, 1 new (total: 1,288) 1 security fixes (total: 158)
Download the new curl release from curl.se as always.
Release presentation
Security
We decided to do a patch release. Then yesterday we got a security vulnerability reported and so now we have that fixed in here as well.
CVE-2024-7264: ASN.1 date parser overread (severity low) libcurl’s ASN1 parser code has the GTime2str() function, used for parsing an ASN.1 Generalized Time field. If given an syntactically incorrect field, the parser might end up using -1 for the length of the time fraction, leading to a strlen() getting performed on a pointer to a heap buffer area that is not (purposely) null terminated.
Bugfixes
This release is done only because we shipped a few regressions in 8.9.0 we rather let users avoid. Here are some noteworthy fixes from the past week:
connection shutdown fix for event based processing – this would cause applications to keep monitoring sockets “too much”, easily leading to busy-loops or worse
cmake builds detect libssh and nettle better
several libcurl functions now survive NULL pointer inputs better
fixed an Apple SDK bug workaround for non-macOS targets
the curl tool builds with the manual enabled on OS400
works around an IBM (OS400) ASCII run-time library bug
speed limiting for 32bit systems had the wrong math
allow wolfSSL’s implementation of kyber to be used
wolfssl CA store caching fix
more defensive and portable socket code for the curl tool’s --ip-tos logic
the 258th release 11 changes 63 days (total: 9,623) 260 bugfixes (total: 10,531) 423 commits (total: 32,704) 0 new public libcurl function (total: 94) 1 new curl_easy_setopt() option (total: 306) 4 new curl command line option (total: 263) 80 contributors, 38 new (total: 3,209) 47 authors, 16 new (total: 1,288) 2 security fixes (total: 157)
Download the new curl release from curl.se as always.
Release presentation
Security
Today we fix two security vulnerabilities and publish all details about them.
CVE-2024-6197: freeing stack buffer in utf8asn1str. (severity medium) libcurl’s ASN1 parser has this utf8asn1str() function used for parsing an ASN.1 UTF-8 string. It can detect an invalid field and return error. Unfortunately, when doing so it also invokes free() on a 4 byte local stack buffer.
CVE-2024-6874: macidn punycode buffer overread. (severity low) libcurl’s URL API function curl_url_get() offers punycode conversions, to and from IDN. Asking to convert a name that is exactly 256 bytes, libcurl ends up reading outside of a stack based buffer when built to use the macidn IDN backend. The conversion function then fills up the provided buffer exactly – but does not null terminate the string.
Changes
–ip-tos (IP Type of Service / Traffic Class). Lets users set this IP header field to a number.
–mptcp. Asks curl to enable the Multipath TCP option for this connection, which if the server also allows it may make the TCP connection to go over multiple network paths.
–vlan-priority. Makes curl set the VLAN priority field for its IP traffic. This is typically a field used in the network layer below IP (think Ethernet), so it is not likely to survive through IP routers. A local network thing.
–keepalive-cnt (and CURLOPT_TCP_KEEPCNT). Specify how many keeplive probes curl should send before it considers the connection to be dead.
–write-out ‘%{num_retries}’ is a new variable for the info output that outputs the number of retries that were done for the previous transfer (when –retry was used).
gnutls now supports CA caching. For libcurl using applications, this can really speed up doing serial TLS connections.
mbedtls supports CURLOPT_CERTINFO. Returns certificate information to the application.
noproxy patterns need to be comma separated. Space separation is no longer enough.
In no other release ever before in curl’s long history have there been this many bugfixes: 260. Some of my favorites are:
cmake: 26 separate bugfixes
configure: 10 separate bugfixes
–help category cleanup and list categories in –help
allow etag and content-disposition for 3xx reply
docs: countless fixes, polish and corections
show name and keywords for failed tests in summary
avoid using GetAddrInfoExW with impersonation
URL encode the canonical path for aws-sigv4
fix DoH cleanup
fix memory leak and zero-length HTTPS RR crash in DoH
allow DoH transfers to override max connection limit
fix ß with AppleIDN
fix compilation with OpenSSL 1.x with md4 disabled
do a final progress update on connect failure
multi: fix pollset during RESOLVING phase
enable UDP GRO for QUIC
require at least OpenSSL 3.3 for QUIC
add shutdown support for HTTP/3 (QUIC)
fix CRLF conversion of input
fixed starttls for SMTP
change TCP keepalive from ms to seconds on DragonFly BSD
support TCP keepalive parameters on Solaris <11.4
shutdown TLS and TCP better
gnutls: pass in SNI name, not hostname when checking cert
gnutls: rectify the TLS version checks for QUIC
mbedtls v3.6.0 workarounds
several x509 asn.1 parser fixes
Next
Because the 8.9.0 release spent an extra week for its release cycle, the next one is going to be one week shorter. We do this by shortening the feature window to just two weeks this time, which might impact how many new features and changes we manage to merge.
We have a large amount of pull requests for changes already pending merge, waiting for the release window to open.
If all goes well, the next release is named 8.10.0 and eventually ships on September 11, 2024.
the 257th release 8 changes 56 days (total: 9,560) 220 bug-fixes (total: 10,271) 348 commits (total: 32,280) 1 new public libcurl function (total: 94) 1 new curl_easy_setopt() option (total: 305) 1 new curl command line option (total: 259) 84 contributors, 41 new (total: 3,173) 49 authors, 20 new (total: 1,272) 0 security fixes (total: 155)
Download the new curl release from curl.se as always.
Release presentation
Security
It feels good to be able to say that this time around we do not have a single security vulnerability to announce and we in fact do not have any in the queue either.
Some of the bugfixes from this cycle that might be worth noticing:
dist and build
reproducible tarballs. I will do a separate post with details later, but now it is easy for anyone who wants to, to generate an identical copy to verify what we ship.
docs/RELEASE-TOOLS.md into the tarball. This documents the tools and versions used to generate the files included in the tarball that are not present in git.
drop MSVC project files for recent versions. If you need to generate them for more recent versions, cmake can do it for you.
configure fix HAVE_IOCTLSOCKET_FIONBIO test for gcc 14. It runs more picky by default so it would always fail the check.
add -q as first option when invoking curl for tests. To reduce the risk of people having a ~/.curlrc file that ruins things.
fix make install with configure –disable-docs
tool
make –help adapt to the terminal width. Makes it easier on the eye when the terminal is wider.
limit rate unpause for -T . uploads. Avoids busy-looping
curl output warning for leading unicode quote character. Because it seems like a fairly common mistake when people copy and paste command lines from random sources
don’t truncate the etag save file by default. A regression less.
TLS
bearssl: use common code for cipher suite lookup
mbedtls: call mbedtls_ssl_setup() after RNG callback is set. Otherwise, more recent versions of mbedTLS will just return error.
mbedtls: support TLS 1.3. If you use a new enough version.
openssl: do not set SSL_MODE_RELEASE_BUFFERS. Uses slightly more memory, but uses fewer memory allocation calls.
wolfssl: plug memory leak in wolfssl_connect_step2()
bindings
openldap: create ldap URLs correctly for IPv6 addresses, doing LDAP with IPv6 numerical IP addresses in the URL just did not work previously.
quiche: expire all active transfers on connection close
quiche: trust its timeout handling
libcurl
fix curl_global_cleanup crash in Windows. A regression coming from the introduction of the async name resolver function.
brotli and others, pass through 0-length writes
ignore duplicate chunked encoding. Apparently some sites do this and browsers let them so we need to let it slide…
CURLINFO_REQUEST_SIZE: fixed
ftp: add tracing support. Gives us better tooling to track down FTP problems.
http2: emit RST when client write fails. Previously it would just silently leave the stream there…
http: reject HTTP major version switch mid connection. This should of course never happen, but if it does, curl will error out correctly.
multi: introduce SETUP state for better timeouts. This adds a proper separation for when the existing transfer is retried or when the state machine is restarted because it make as a new transfer.
multi: timeout handles even without connection. They would previously often be exempted from checks and would linger for too long until stopped.
fix handling of paused upload on completed download
do not URL decode proxy credentials
allow setting port number zero. Remember this old post?
the 255th and 256th releases 5 changes 56 days (total: 9,504) 162 bug-fixes (total: 10,050) 246 commits (total: 31,931) 0 new public libcurl function (total: 93) 0 new curl_easy_setopt() option (total: 304) 0 new curl command line option (total: 258) 92 contributors, 56 new (total: 3,133) 37 authors, 15 new (total: 1,252) 4 security fixes (total: 155)
Versions
I first released 8.7.0, but immediately someone pointed out that one of the files in the tarballs was broken, so I fixed the issue, created a new set of tarballs, bumped the version and uploaded the new set. The new release is 8.7.1 but of course it has the same set of changes. We just pretend we did not upload 8.7.0.
Release presentation
Security
CVE-2024-2004: Usage of disabled protocol. (low) When a protocol selection parameter option disables all protocols without adding any then the default set of protocols would remain in the allowed set due to an error in the logic for removing protocols.
CVE-2024-2398: HTTP/2 push headers memory-leak. (medium) When an application tells libcurl it wants to allow HTTP/2 server push, and the amount of received headers for the push surpasses the maximum allowed limit (1000), libcurl aborts the server push. When aborting, libcurl inadvertently does not free all the previously allocated headers and instead leaks the memory.
CVE-2024-2379: QUIC certificate check bypass with wolfSSL. (low) libcurl skips the certificate verification for a QUIC connection under certain conditions, when built to use wolfSSL. If told to use an unknown/bad cipher or curve, the error path accidentally skips the verification and returns OK, thus ignoring any certificate problems.
CVE-2024-2466: TLS certificate check bypass with mbedTLS. (medium) libcurl did not check the server certificate of TLS connections done to a host specified as an IP address, when built to use mbedTLS.
Changes
configure: add –disable-docs flag. This skips the step generating the manpages, which for many people is unnecessary.
CURLINFO_USED_PROXY: return bool whether the proxy was used. Useful when having a filter that only lets some transfers use the proxy.
write-out: add ‘%{proxy_used}’. The same as above but for the tool.
digest: support SHA-512/256. Support more modern digest authentication.
DoH: add trace configuration. Now you get more DoH tracing/logging using the general trace mechanism.
Bugfixes
Some of the bugfixes from this cycle that might be worth noticing:
configure: find libpsl with pkg-config. Makes configure better at finding libpsl and making use of the correct flags and sub-dependencies when linking with it.
configure: find rustls with pkg-config. Similar adjustment but for rustls.
cookie: if psl fails, reject the cookie. A run-time failure should not allow the cookie through.
curl: exit on config file parser errors. We can insist on the config file to be correct as otherwise something unintended might go through.
curl: make –libcurl output better CURLOPT_*SSLVERSION. This option takes a bitmask made out of two separate enum ranges.
file: use xfer buf for file:// transfers. The main effect being that it can use a larger buffer which can make faster transfers.
http: better error message for HTTP/1.x response without status line
https-proxy: use IP address and cert with ip in alt names. Connecting to a HTTPS proxy using an IP address with a URL also using an IP address and those addresses were different versions, curl would get it wrong.
mprintf: fix format prefix I32/I64 for windows compilers
OpenSSL QUIC: adapt to v3.3.x. Pending improvements in OpenSSL is going to enhance curl’s ability to do HTTP/3 using it.
paramhlp: fix CRLF-stripping files with “-d @file”. curl would do wrong for line ending consisting of CR only
rustls: make curl compile with 0.12.0. Adjusted to use the modified APIs.
schannel: fix hang on unexpected server close
sendf: ignore response body to HEAD. A regression made curl complain if a HEAD request would get body data.
smtp: fix STARTTLS. Another regression fixed.
strtoofft: fix the overflow check. The previous overflow check was relying on undefined behavior. This is in code only for platforms without a proper native parser for 64 bit sized numbers.
TLS: start shutdown only when peer did not already close.
curl: only parse etag + content-disposition for 2xx.
curl: accept a blank -w “”
curl: handle non-existing (out of range) short-options
curl: change precedence of server Retry-After time
curl: shorter –help texts. With some polish to make the output look nicer, in particular “curl –help all”.
transfer.c: break receive loop in speed limited transfers, To make libcurl adapt more precisely to the network speed limit set by the application.
the 254th release 7 changes 56 days (total: 9,448) 154 bug-fixes (total: 9,888) 257 commits (total: 31,684) 0 new public libcurl function (total: 93) 1 new curl_easy_setopt() option (total: 304) 0 new curl command line option (total: 258) 65 contributors, 40 new (total: 3,078) 36 authors, 18 new (total: 1,237) 1 security fix (total: 151)
Release presentation
Security
CVE-2024-0853: OCSP verification bypass with TLS session reuse. curl inadvertently kept the SSL session ID for connections in its cache even when the verify status (OCSP stapling) test failed. A subsequent transfer to the same hostname could then succeed if the session ID cache was still fresh, which then skipped the verify status check.
Changes
Markdown documentation. Most of the libcurl and command line documentation is now written using (basic) markdown instead of previous formats. Easier to read, easier to write.
CURLE_TOO_LARGE. A new libcurl error code for when “something” is growing too big to be allowed. Like a URL, a HTTP request or similar. Previously it would return out of memory for those situations which caused confusion to users.
CURLINFO_QUEUE_TIME_T. Applications can now ask libcurl how long a transfer was “queued” internally before it actually started.
CURLOPT_SERVER_RESPONSE_TIMEOUT_MS. A new millisecond version of the already existing option to allow applications higher resolution control.
Use GetAddrInfoExW on >= Windows 8. On current Windows versions libcurl will now do asynchronous name resolving by default without using threads, which should be less resource heavy.
libpsl detection failure in configure causes error. If configure cannot find libpsl it will require the user to say that it should not be used, or to fix the problem. To make people who build curl more aware of the PSL state of the build.
runtests supports -gl, When you invoke individual test cases on macOS, you can now ask to run it with lldb with -gl just as you have been able to run it with gdb using -g for decades. Helps debugging difficult cases.
Bugfixes
Here some of my favorite bugfixes from this cycle:
configure: add libngtcp2_crypto_boringssl detection. Previously it would only detect and build out of the box with the quictls version of ngtcp2 builds.
configure: when enabling QUIC, check that TLS supports QUIC. More efforts trying to detect wrong and invalid build combinations earlier, to avoid users ending up with broken builds.
all libcurl man page examples are verified in CI.Every man page example now compiles cleanly. This step made us detect and fix numerous tiny mistakes of the most annoying kind: when you copy code from docs and it does not work.
curl shows ipfs and ipns as supported “protocols”. In the regular --version output. Even if they are converted to https:// internally.
curl bsearches command line options. The command line parsing is now magnitudes faster. Of course it will not really be noticeable outside of the most extreme cases.
curl stopped supporting @filename style for --cookie. This syntax was never documented and was not used in any test case. It was risking to cause unwanted surprises.
curl –remove-on-error only removes “real” files. Mostly as a precaution for when users are unclever enough to run curl with elevated privileges and would save to a device or named pipe etc.
curl no longer sets the file comment on Amiga. It would truncate the URL weirdly and also risk leak credentials if such were used in the URL.
lib: reduced use of the download buffer all over. The download buffer has over time been abused for all kinds of buffer purposes. This cycle we have made a lot of such buffer use instead start use their own buffers. With a little luck, this will make us possible use a single download buffer for all transfer in a multi handle, thereby drastically reducing the amount of used memory when doing parallel transfers. With no behavior difference or performance degradation. Details on this will follow later.
lib: use memdup0 instead of malloc + memcpy. This was a common code pattern, and with this we reduce the number of mallocs and memcpys at the same time – which we think is good since they are known “problem functions” that are easy to mess up.
lib: various conversions from malloc to dynbuf. In similar spirit as the above, we continued to switch more functions away from using malloc and family to instead use the internal dynbuf API for managing dynamic buffers in a way that is less likely to cause memory related issues.
resolving: with modern c-ares, use its default timeout. It means tighter timeouts by default but also that this combo now also respect the timeout that can be specified in resolve.conf.
headers API: make sure the trailing newline is never stored. A header with no content on the right side of the colon would erroneously get its trailing newline stored as content
mprintf: overhaul, performance and bugfixes. Now the curl printf functions work even more similar to the glibc counterparts especially when provided illegal %-combinations and when using the <num>$ operator. Performance measurements on this new code also says this code now executes around 30% faster on commonly used format strings.
ftp: handle the PORT parsing without allocation. Minor cleanup.
http: check for “Host:” case insensitively. If you would ask to disable this header with a different casing that what was compared, it would still send an empty header in the request.
http: only act on 101 responses when they are HTTP/1.1. If a HTTP response says another protocol version with a 101 response, it is now considered an illegal combination.
openldap: fix an LDAP crash. LDAP without TLS would crash on basic use.
openldap: fix STARTTLS. It was recently broken in a refactor.
Next
There are no revolutionary changes in the pipe, but there are a series of things we most likely are going to land in the next cycle making the next version number likely to become 8.7.0.
