Numbers

the 258th release
11 changes
63 days (total: 9,623)
260 bugfixes (total: 10,531)
423 commits (total: 32,704)
0 new public libcurl function (total: 94)
1 new curl_easy_setopt() option (total: 306)
4 new curl command line option (total: 263)
80 contributors, 38 new (total: 3,209)
47 authors, 16 new (total: 1,288)
2 security fixes (total: 157)

Download the new curl release from curl.se as always.

Release presentation

At 10:00 CEST (08:00 UTC) I will do a live-streamed release presentation of curl 8.9.0 on twitch. Afterwards, this paragraph will be replaced with a link to the recorded video of it.

Security

Today we fix two security vulnerabilities and publish all details about them.

• CVE-2024-6197: freeing stack buffer in utf8asn1str. (severity medium) libcurl’s ASN1 parser has this utf8asn1str() function used for parsing an ASN.1 UTF-8 string. It can detect an invalid field and return error. Unfortunately, when doing so it also invokes free() on a 4 byte local stack buffer.
• CVE-2024-6874: macidn punycode buffer overread. (severity low) libcurl’s URL API function curl_url_get() offers punycode conversions, to and from IDN. Asking to convert a name that is exactly 256 bytes, libcurl ends up reading outside of a stack based buffer when built to use the macidn IDN backend. The conversion function then fills up the provided buffer exactly – but does not null terminate the string.

Changes

• –ip-tos (IP Type of Service / Traffic Class). Lets users set this IP header field to a number.
• –mptcp. Asks curl to enable the Multipath TCP option for this connection, which if the server also allows it may make the TCP connection to go over multiple network paths.
• –vlan-priority. Makes curl set the VLAN priority field for its IP traffic. This is typically a field used in the network layer below IP (think Ethernet), so it is not likely to survive through IP routers. A local network thing.
• –keepalive-cnt (and CURLOPT_TCP_KEEPCNT). Specify how many keeplive probes curl should send before it considers the connection to be dead.
• –write-out ‘%{num_retries}’ is a new variable for the info output that outputs the number of retries that were done for the previous transfer (when –retry was used).
• gnutls now supports CA caching. For libcurl using applications, this can really speed up doing serial TLS connections.
• mbedtls supports CURLOPT_CERTINFO. Returns certificate information to the application.
• noproxy patterns need to be comma separated. Space separation is no longer enough.
• Support binding a connection to both interface and IP, not just one of them.
• The URL API added CURLU_NO_GUESS_SCHEME, to allow an application to figure out if the scheme for a previously parsed URL was set or guessed.
• wolfssl now supports CA caching

Bugfixes

In no other release ever before in curl’s long history have there been this many bugfixes: 260. Some of my favorites are:

• cmake: 26 separate bugfixes
• configure: 10 separate bugfixes
• –help category cleanup and list categories in –help
• allow etag and content-disposition for 3xx reply
• docs: countless fixes, polish and corections
• show name and keywords for failed tests in summary
• avoid using GetAddrInfoExW with impersonation
• URL encode the canonical path for aws-sigv4
• fix DoH cleanup
• fix memory leak and zero-length HTTPS RR crash in DoH
• allow DoH transfers to override max connection limit
• fix ß with AppleIDN
• fix compilation with OpenSSL 1.x with md4 disabled
• do a final progress update on connect failure
• multi: fix pollset during RESOLVING phase
• enable UDP GRO for QUIC
• require at least OpenSSL 3.3 for QUIC
• add shutdown support for HTTP/3 (QUIC)
• fix CRLF conversion of input
• fixed starttls for SMTP
• change TCP keepalive from ms to seconds on DragonFly BSD
• support TCP keepalive parameters on Solaris <11.4
• shutdown TLS and TCP better
• gnutls: pass in SNI name, not hostname when checking cert
• gnutls: rectify the TLS version checks for QUIC
• mbedtls v3.6.0 workarounds
• several x509 asn.1 parser fixes

Next

Because the 8.9.0 release spent an extra week for its release cycle, the next one is going to be one week shorter. We do this by shortening the feature window to just two weeks this time, which might impact how many new features and changes we manage to merge.

We have a large amount of pull requests for changes already pending merge, waiting for the release window to open.

If all goes well, the next release is named 8.10.0 and eventually ships on September 11, 2024.

Numbers

the 257th release
8 changes
56 days (total: 9,560)
220 bug-fixes (total: 10,271)
348 commits (total: 32,280)
1 new public libcurl function (total: 94)
1 new curl_easy_setopt() option (total: 305)
1 new curl command line option (total: 259)
84 contributors, 41 new (total: 3,173)
49 authors, 20 new (total: 1,272)
0 security fixes (total: 155)

Download the new curl release from curl.se as always.

Release presentation

Security

It feels good to be able to say that this time around we do not have a single security vulnerability to announce and we in fact do not have any in the queue either.

Changes

• curl_version_info() provides librtmp version
• file:// supports directory listings
• AppleIDN support for macOS/iOS
• add curl_multi_waitfds
• mbedTLS supports CURLOPT_SSL_CIPHER_LIST
• drop support for NTLM_WB
• experimental ECH (Encrypted Client Hello)
• add CURLU_GET_EMPTY for empty queries and fragments

Bugfixes

Some of the bugfixes from this cycle that might be worth noticing:

dist and build

• reproducible tarballs. I will do a separate post with details later, but now it is easy for anyone who wants to, to generate an identical copy to verify what we ship.
• docs/RELEASE-TOOLS.md into the tarball. This documents the tools and versions used to generate the files included in the tarball that are not present in git.
• drop MSVC project files for recent versions. If you need to generate them for more recent versions, cmake can do it for you.
• configure fix HAVE_IOCTLSOCKET_FIONBIO test for gcc 14. It runs more picky by default so it would always fail the check.
• add -q as first option when invoking curl for tests. To reduce the risk of people having a ~/.curlrc file that ruins things.
• fix make install with configure –disable-docs

tool

• make –help adapt to the terminal width. Makes it easier on the eye when the terminal is wider.
• limit rate unpause for -T . uploads. Avoids busy-looping
• curl output warning for leading unicode quote character. Because it seems like a fairly common mistake when people copy and paste command lines from random sources
• don’t truncate the etag save file by default. A regression less.

TLS

• bearssl: use common code for cipher suite lookup
• mbedtls: call mbedtls_ssl_setup() after RNG callback is set. Otherwise, more recent versions of mbedTLS will just return error.
• mbedtls: support TLS 1.3. If you use a new enough version.
• openssl: do not set SSL_MODE_RELEASE_BUFFERS. Uses slightly more memory, but uses fewer memory allocation calls.
• wolfssl: plug memory leak in wolfssl_connect_step2()

bindings

• openldap: create ldap URLs correctly for IPv6 addresses, doing LDAP with IPv6 numerical IP addresses in the URL just did not work previously.
• quiche: expire all active transfers on connection close
• quiche: trust its timeout handling

libcurl

• fix curl_global_cleanup crash in Windows. A regression coming from the introduction of the async name resolver function.
• brotli and others, pass through 0-length writes
• ignore duplicate chunked encoding. Apparently some sites do this and browsers let them so we need to let it slide…
• CURLINFO_REQUEST_SIZE: fixed
• ftp: add tracing support. Gives us better tooling to track down FTP problems.
• http2: emit RST when client write fails. Previously it would just silently leave the stream there…
• http: reject HTTP major version switch mid connection. This should of course never happen, but if it does, curl will error out correctly.
• multi: introduce SETUP state for better timeouts. This adds a proper separation for when the existing transfer is retried or when the state machine is restarted because it make as a new transfer.
• multi: timeout handles even without connection. They would previously often be exempted from checks and would linger for too long until stopped.
• fix handling of paused upload on completed download
• do not URL decode proxy credentials
• allow setting port number zero. Remember this old post?
• fix relative redirects to fragment-only
• fix memory leak in websocket error path

Numbers

the 255th and 256th releases
5 changes
56 days (total: 9,504)
162 bug-fixes (total: 10,050)
246 commits (total: 31,931)
0 new public libcurl function (total: 93)
0 new curl_easy_setopt() option (total: 304)
0 new curl command line option (total: 258)
92 contributors, 56 new (total: 3,133)
37 authors, 15 new (total: 1,252)
4 security fixes (total: 155)

Versions

I first released 8.7.0, but immediately someone pointed out that one of the files in the tarballs was broken, so I fixed the issue, created a new set of tarballs, bumped the version and uploaded the new set. The new release is 8.7.1 but of course it has the same set of changes. We just pretend we did not upload 8.7.0.

Release presentation

Security

• CVE-2024-2004: Usage of disabled protocol. (low)
  When a protocol selection parameter option disables all protocols without adding any then the default set of protocols would remain in the allowed set due to an error in the logic for removing protocols.
• CVE-2024-2398: HTTP/2 push headers memory-leak. (medium)
  When an application tells libcurl it wants to allow HTTP/2 server push, and the amount of received headers for the push surpasses the maximum allowed limit (1000), libcurl aborts the server push. When aborting, libcurl inadvertently does not free all the previously allocated headers and instead leaks the memory.
• CVE-2024-2379: QUIC certificate check bypass with wolfSSL. (low)
  libcurl skips the certificate verification for a QUIC connection under certain conditions, when built to use wolfSSL. If told to use an unknown/bad cipher or curve, the error path accidentally skips the verification and returns OK, thus ignoring any certificate problems.
• CVE-2024-2466: TLS certificate check bypass with mbedTLS. (medium)
  libcurl did not check the server certificate of TLS connections done to a host specified as an IP address, when built to use mbedTLS.

Changes

• configure: add –disable-docs flag. This skips the step generating the manpages, which for many people is unnecessary.
• CURLINFO_USED_PROXY: return bool whether the proxy was used. Useful when having a filter that only lets some transfers use the proxy.
• write-out: add ‘%{proxy_used}’. The same as above but for the tool.
• digest: support SHA-512/256. Support more modern digest authentication.
• DoH: add trace configuration. Now you get more DoH tracing/logging using the general trace mechanism.

Bugfixes

Some of the bugfixes from this cycle that might be worth noticing:

• configure: find libpsl with pkg-config. Makes configure better at finding libpsl and making use of the correct flags and sub-dependencies when linking with it.
• configure: find rustls with pkg-config. Similar adjustment but for rustls.
• cookie: if psl fails, reject the cookie. A run-time failure should not allow the cookie through.
• curl: exit on config file parser errors. We can insist on the config file to be correct as otherwise something unintended might go through.
• curl: make –libcurl output better CURLOPT_*SSLVERSION. This option takes a bitmask made out of two separate enum ranges.
• file: use xfer buf for file:// transfers. The main effect being that it can use a larger buffer which can make faster transfers.
• http: better error message for HTTP/1.x response without status line
• https-proxy: use IP address and cert with ip in alt names. Connecting to a HTTPS proxy using an IP address with a URL also using an IP address and those addresses were different versions, curl would get it wrong.
• mprintf: fix format prefix I32/I64 for windows compilers
• OpenSSL QUIC: adapt to v3.3.x. Pending improvements in OpenSSL is going to enhance curl’s ability to do HTTP/3 using it.
• paramhlp: fix CRLF-stripping files with “-d @file”. curl would do wrong for line ending consisting of CR only
• rustls: make curl compile with 0.12.0. Adjusted to use the modified APIs.
• schannel: fix hang on unexpected server close
• sendf: ignore response body to HEAD. A regression made curl complain if a HEAD request would get body data.
• smtp: fix STARTTLS. Another regression fixed.
• strtoofft: fix the overflow check. The previous overflow check was relying on undefined behavior. This is in code only for platforms without a proper native parser for 64 bit sized numbers.
• TLS: start shutdown only when peer did not already close.
• curl: only parse etag + content-disposition for 2xx.
• curl: accept a blank -w “”
• curl: handle non-existing (out of range) short-options
• curl: change precedence of server Retry-After time
• curl: shorter –help texts. With some polish to make the output look nicer, in particular “curl –help all”.
• transfer.c: break receive loop in speed limited transfers, To make libcurl adapt more precisely to the network speed limit set by the application.

Numbers

the 254th release
7 changes
56 days (total: 9,448)
154 bug-fixes (total: 9,888)
257 commits (total: 31,684)
0 new public libcurl function (total: 93)
1 new curl_easy_setopt() option (total: 304)
0 new curl command line option (total: 258)
65 contributors, 40 new (total: 3,078)
36 authors, 18 new (total: 1,237)
1 security fix (total: 151)

Release presentation

Security

CVE-2024-0853: OCSP verification bypass with TLS session reuse. curl inadvertently kept the SSL session ID for connections in its cache even when the verify status (OCSP stapling) test failed. A subsequent transfer to the same hostname could then succeed if the session ID cache was still fresh, which then skipped the verify status check.

Changes

• Markdown documentation. Most of the libcurl and command line documentation is now written using (basic) markdown instead of previous formats. Easier to read, easier to write.

• CURLE_TOO_LARGE. A new libcurl error code for when “something” is growing too big to be allowed. Like a URL, a HTTP request or similar. Previously it would return out of memory for those situations which caused confusion to users.

• CURLINFO_QUEUE_TIME_T. Applications can now ask libcurl how long a transfer was “queued” internally before it actually started.

• CURLOPT_SERVER_RESPONSE_TIMEOUT_MS. A new millisecond version of the already existing option to allow applications higher resolution control.

• Use GetAddrInfoExW on >= Windows 8. On current Windows versions libcurl will now do asynchronous name resolving by default without using threads, which should be less resource heavy.

• libpsl detection failure in configure causes error. If configure cannot find libpsl it will require the user to say that it should not be used, or to fix the problem. To make people who build curl more aware of the PSL state of the build.

• runtests supports -gl, When you invoke individual test cases on macOS, you can now ask to run it with lldb with -gl just as you have been able to run it with gdb using -g for decades. Helps debugging difficult cases.

Bugfixes

Here some of my favorite bugfixes from this cycle:

configure: add libngtcp2_crypto_boringssl detection. Previously it would only detect and build out of the box with the quictls version of ngtcp2 builds.

configure: when enabling QUIC, check that TLS supports QUIC. More efforts trying to detect wrong and invalid build combinations earlier, to avoid users ending up with broken builds.

all libcurl man page examples are verified in CI. Every man page example now compiles cleanly. This step made us detect and fix numerous tiny mistakes of the most annoying kind: when you copy code from docs and it does not work.

curl shows ipfs and ipns as supported “protocols”. In the regular --version output. Even if they are converted to https:// internally.

curl bsearches command line options. The command line parsing is now magnitudes faster. Of course it will not really be noticeable outside of the most extreme cases.

curl stopped supporting @filename style for --cookie. This syntax was never documented and was not used in any test case. It was risking to cause unwanted surprises.

curl –remove-on-error only removes “real” files. Mostly as a precaution for when users are unclever enough to run curl with elevated privileges and would save to a device or named pipe etc.

curl no longer sets the file comment on Amiga. It would truncate the URL weirdly and also risk leak credentials if such were used in the URL.

lib: reduced use of the download buffer all over. The download buffer has over time been abused for all kinds of buffer purposes. This cycle we have made a lot of such buffer use instead start use their own buffers. With a little luck, this will make us possible use a single download buffer for all transfer in a multi handle, thereby drastically reducing the amount of used memory when doing parallel transfers. With no behavior difference or performance degradation. Details on this will follow later.

lib: use memdup0 instead of malloc + memcpy. This was a common code pattern, and with this we reduce the number of mallocs and memcpys at the same time – which we think is good since they are known “problem functions” that are easy to mess up.

lib: various conversions from malloc to dynbuf. In similar spirit as the above, we continued to switch more functions away from using malloc and family to instead use the internal dynbuf API for managing dynamic buffers in a way that is less likely to cause memory related issues.

resolving: with modern c-ares, use its default timeout. It means tighter timeouts by default but also that this combo now also respect the timeout that can be specified in resolve.conf.

headers API: make sure the trailing newline is never stored. A header with no content on the right side of the colon would erroneously get its trailing newline stored as content

mprintf: overhaul, performance and bugfixes. Now the curl printf functions work even more similar to the glibc counterparts especially when provided illegal %-combinations and when using the <num>$ operator. Performance measurements on this new code also says this code now executes around 30% faster on commonly used format strings.

ftp: handle the PORT parsing without allocation. Minor cleanup.

http3: initial support for OpenSSL 3.2 QUIC stack. The forth QUIC backend in curl is here.

http: check for “Host:” case insensitively. If you would ask to disable this header with a different casing that what was compared, it would still send an empty header in the request.

http: only act on 101 responses when they are HTTP/1.1. If a HTTP response says another protocol version with a 101 response, it is now considered an illegal combination.

openldap: fix an LDAP crash. LDAP without TLS would crash on basic use.

openldap: fix STARTTLS. It was recently broken in a refactor.

Next

There are no revolutionary changes in the pipe, but there are a series of things we most likely are going to land in the next cycle making the next version number likely to become 8.7.0.

Release presentation

Numbers

the 253rd release
2 changes
56 days (total: 9,392)
188 bug-fixes (total: 9,734)
266 commits (total: 31,427)
0 new public libcurl function (total: 93)
0 new curl_easy_setopt() option (total: 303)
0 new curl command line option (total: 258)
78 contributors, 43 new (total: 3,039)
40 authors, 19 new (total: 1,219)
2 security fixes (total: 150)

Security

cookie mixed case PSL bypass

(CVE-2023-46218) This flaw allows a malicious HTTP server to set “super cookies” in curl that are then passed back to more origins than what is otherwise allowed or possible. This allows a site to set cookies that then would get sent to different and unrelated sites and domains.

It could do this by exploiting a mixed case flaw in curl’s function that verifies a given cookie domain against the Public Suffix List (PSL). For example a cookie could be set with domain=co.UK when the URL used a lower case hostname curl.co.uk, even though co.uk is listed as a PSL domain.

HSTS long file name clears contents

(CVE-2023-46219) When saving HSTS data to an excessively long file name, curl could end up removing all contents, making subsequent requests using that file unaware of the HSTS status they should otherwise use.

Changes

We have only logged two changes for this cycle.

gnutls supports CURLSSLOPT_NATIVE_CA

If you use libcurl built to use GnuTLS, you too can now set this bit and get to use the Windows CA store directly from within libcurl instead of having to provide a separate PEM file. When your application runs on Windows. libcurl already previously support this for OpenSSL and wolfSSL.

See the CURLOPT_SSL_OPTIONS documentation for details.

HTTP3 with ngtcp2 no longer experimental

The first HTTP/3 code in curl was merged into git in 2019. Now we ship HTTP/3 support non-experimental for the first time. curl supports HTTP/3 using several different backends but this news is only for HTTP/3 built to use ngtcp2 + nghttp3. The other HTTP/3 backends remain experimental for the time being.

Note that in order to be able to use ngtcp2 you need to build with a TLS library that offers the necessary API. This means you need to use one of the following libraries: wolfSSL, quictls, BoringSSL, libressl, AWS-LC or GnuTLS. Let me stress that OpenSSL is not in that list.

Bugfixes

As usual, here I have collected a few of my favorite fixes from this cycle.

improved IPFS and IPNS URL support

Turned out there were some IPFS URLs that curl did not manage properly.

doh: use PIPEWAIT when HTTP/2 is attempted

This makes curl prefer doing DoH requests multiplexed over a single connection rather than doing them as two separate connections. Most DoH lookups do one request for an IPv4 response and a separate one for IPv6.

duphandle: several OOM cleanups

If libcurl runs out of memory in the middle of the curl_easy_duphandle function, it could previously do several mistakes, including free-twice.

hostip: show the list of IPs when resolving is done

The verbose mode now shows the full list of IP addresses that was resolved from the name, before it continues to try to connect to them in a serial fashion.

aws-sigv4: canonicalize valueless query params

Turns out there was yet another glitch in the sigv4 logic that made curl send the wrong checksum for URLs using “valueless” query parameters.

hyper: temporarily remove HTTP/2 support

The hyper integration for HTTP/2 was incorrect and has therefore been disabled for now. hyper support in curl is still experimental.

drop vc10, vc11 and vc12 projects from dist, add vc14.20

We generate less project files for older Visual Studio versions but we added one for a recent version. Going forward, we might soon drop them completely from the tarballs since they can now be generated with cmake.

openssl: include SIG and KEM algorithms in verbose

curl now presents more details from the TLS handshake in the verbose output when using OpenSSL.

openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs

The keyword here is intermediate. Previously, curl would ignore those which would lead to handshake errors because that is not what users expect when you use the native CA store.

openssl: when a session-ID is reused, skip OCSP stapling

When trying to do verify status, the more technical name for OCSP stapling, after a connection has been established using a session-ID, it would return error.

make SOCKS5 use the CURLOPT_IPRESOLVE choice

There are just too many combinations, but we find it likely that a user that asks for a specific IP version probably also wants it for the traffic over the SOCKS proxy.

tool: support bold headers in Windows

The feature that has been provided on other systems since 2018 now comes to Windows.

make the carriage return fit wide progress bars

The progress bar set with -#/--progress-bar had its math off by one, which made it not output the carriage return character when the terminal width was wider than 256 columns, making the output wrong.

url: find scheme with a “perfect hash”

The internal function that scans the list of supported URL schemes no longer iterates through the list but instead uses a “perfect hash”. Although much faster, that was hardly a function that caused performance problems before so it is not likely to actually be measurable.

use ALPN “http/1.1” for HTTP/1.x, including HTTP/1.0

To interoperate better with legacy servers, curl now sends http/1.1 in the ALPN field even when it wants to speak HTTP/1.0. This, because http/1.1 and h2 were the original ALPN codes and some of the old servers that support ALPN from those days don’t know about http/1.0 for ALPN.

all libcurl man pages examples compile cleanly

All (almost five hundred) libcurl man pages have EXAMPLE sections showing their use. Starting now, all of those sections are test-compiled as part of the CI builds to catch mistakes.

Welcome to this new curl release!

Release video

Numbers

the 251st release
9 changes
49 days (total: 9,308)
174 bug-fixes (total: 9,415)
296 commits (total: 30,942)
1 new public libcurl function (total: 92)
0 new curl_easy_setopt() option (total: 303)
2 new curl command line option (total: 257)
80 contributors, 50 new (total: 2,977)
40 authors, 20 new (total: 1,193)
1 security fix (total: 146)

Numbers notes:

1. the release counter now also includes project releases done before the name was changed to curl.
2. The number of security fixes is adjusted due to the recently rejected CVE-2023-32001

Security

We publish a security advisory in association with today’s release.

HTTP headers eat all memory

[CVE-2023-38039] When curl retrieves an HTTP response, it stores the incoming headers so that they can be accessed later via the libcurl headers API.

However, curl did not have a limit in how many or how large headers it would accept in a response, allowing a malicious server to stream an endless series of headers and eventually cause curl to run out of heap memory.

Changes

curl: make %output{} in -w specify a file to write to

The super handy option –write-out become even more convenient now as it can redirect its output into a specific file and not just stdout and stderr.

curl: add “variable” support

The new variable concept now only lets users use environment variables on config files but also opens up for new ways to use curl command lines effectively.

remove gskit support

The gskit TLS library is no longer a provided option when building curl.

remove NSS support

The NSS TLS library is no longer a provided option when building curl. curl still supports building with twelve different TLS libraries even after the removal of these two.

configure –disable-bindlocal builds curl without local binding support

As a next step in the gradual movement to allow more and more features to get enabled/disabled at build time, the time came to the bindlocal function, which is the feature that binds the local end of a connection. Primarily intended for tiny-curl purposes when you aim for a minimal footprint build.

make tracing available in non-debug builds

Starting now, libcurl offers curl_global_trace and curl offers –trace-config to ask for what specific details to include in the verbose logging output. This is a way for a non-debug build to provide more protocol level details from transfers in ways that were previously not possible. Allows for users to report bugs better and provide more insights from real-world problematic scenarios.

CURLOPT_MAXREDIRS defaults to 30

As a precaution, we change the default from unlimited to 30.

CURLU_PUNY2IDN – convert punycode to IDN

The URL API gets the ability to convert to an International Domain Name when given a punycode version. Previously it could only do the conversion in the other direction.

wolfssl: support loading system CA certificates

curl built with wolfSSL now can use the “native CA” option which then makes it possible to use the native CA store on several platforms instead of using a separately provided external file.

Bugfixes

More than 160 bugfixes are logged for this release, but here are a few selected highlights.

accept and parse IPv6 addresses in alt-svc response headers

Previously curl would not parse and accept such hosts.

c-ares: reduce timeout to 2000ms

The default c-ares DNS timeout is set to the same time that c-ares itself has changed to in their next pending release.

make CURLOPT_HAPROXY_CLIENT_IP set the source IP

It was wrongly set as destination instead of source.

cmake: ten separate improvements

Numerous smaller and larger fixes that made the cmake build of curl several notches better.

stop halving the remaining connect timeout when less than 600 ms left

When curl connects to a host that resolves to multiple IP addresses, it allows half the timeout time for the current IP before it moves on to attempt the next IP in the list. That “halving” is now stopped when there is less than 600 milliseconds left to reduce problems with too short times.

docs: rewrite to present tense

Most of the curl documentation now says “this option does this” instead of “this option will do this”

escape all dashes (ASCII minus) to avoid Unicode hyphens in curl.1 man page

It turns out the curl man page as generated previously, would make the man command use a Unicode hyphen instead of ASCII minus when displayed. This broke copy and paste and it made it impossible to properly search for minus/dash when viewing the man page.

accept leading whitespace on first HTTP response header

curl is now less strict if the first HTTP/1 response header starts with space or tab, thus looking like it is a “fold” when it not. Other commonly used tools/browsers accept this kind of bad syntax and so does curl now.

avoid too early HTTP/2 connection re-use/multiplexing

When doing lots of parallel transfers curl might need to create a second connection when the first reaches its maximum number of streams. In that situation, curl would try to multiplex on that new connection too early, already before it was properly setup and be ready for use, leading to transfer errors.

http/http2/http3: fix sending large requests

Logic for all supported HTTP versions had (different) issues in handling sending very large requests.

aws-sigv4: canonicalize the query

Using aws-sigv4 authentication would fail if the query part was not manually crafted to be correct: sorted, uppercase %-encoding and all the name/value pairs alpha-sorted. Now curl does this itself.

make aws-sigv4 not require TLS to be used

The –aws-sigv4 option no longer requires an HTTPS:// URL to be used.

lib: move mimepost data from ->req.p.http to ->state

The moving of internal data from one struct to another made data survive between two requests and thus fixed a bug involving redirects with MIMEPOST that needed to rewind.

use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set

Turns out curl would still resolve both IPv4 and IPv6 names even if ipv6-only connections were being requested, thus getting some extra names in vein.

system.h: add CURL_OFF_T definitions on HP-UX with HP aCC

Starting now, curl builds properly on more HP-UX machines.

tests: update cookie expiry dates to far in the future

curl’s test suite now runs fine even when executed in a year after 2038.

tool_filetime: make -z work with file dates before 1970

The -z option can get the file date off a local file and use that in a HTTP time condition request, but if the file was older than January 1 1970 it would act wrongly.

transfer: also stop the sending on closed connection

When curl sent off a HTTP/1 request and the connection was closed before the sending was complete, curl could end up not detecting that and ending the transfer correctly.

don’t set TIMER_STARTTRANSFER on first send

Adjustments were made to make this timestamp work as actually documented.

make zoneid duplicated in curl_url_dup

This dup function did not correctly duplicate the zone id from the source handle, making it an incomplete duplicate.

quic: don’t set SNI if hostname is an IP address

curl would wrongly populate the SNI field with the IP address when doing QUIC connections to such.

Next

This is a dot-zero release. If there are any important enough regressions shipped in this version, we will do a follow-up release within shortly. Report all and any problems you spot.

Welcome. Due to some annoying regressions in the previous release we think we owned it to everyone to do a quick patch follow-up.

Download curl 8.2.1 from https://curl.se.

Release presentation

Numbers

the 221st release
0 changes
7 days (total: 9,259)
27 bug-fixes (total: 9,194)
37 commits (total: 30,646)
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 303)
0 new curl command line option (total: 255)
20 contributors, 7 new (total: 2,927)
13 authors, 3 new (total: 1,173)
0 security fixes (total: 146)

Bugfixes

Here are some the most important fixes in this release

configure: check for nghttp2_session_get_stream_local_window_size

We use this function now, introduced in nghttp2 1.15.0, released in September 2016.

return IPv6 first for localhost resolves

Resolving “localhost” did not return the (fixed) addresses in the correct order. It now returns IPv6 as the first.

http2 regression on upload EOF handling

When we added an optimization in the previous release we missed a code path that sometimes lead to “hanging” uploads over HTTP/2.

os400: correct EXPECTED_STRING_LASTZEROTERMINATED

curl builds fine for “IBM i” again.

quiche: fix lookup of transfer at multi

Doing multiplexed HTTP/3 over multiple connections with quiche works much better.

mkhelp: strip off escape sequences

The command sequence that generates the man page display code for the --manual option did at some point regress to include escape sequences. Now those sequences are properly filtered out.

fix build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T

A build problem was fixed for these rare platforms.

do not clear the credentials on redirect to absolute URL

Yet another regression that we allowed because we apparently did not have a test for this! Now we have a test and redirects to the same origin when using -u for credentials now send the credentials even in the redirected request.

Welcome to another curl release. You know how this dance goes…

Numbers

the 220th release
5 changes
50 days (total: 9,252)
122 bug-fixes (total: 9,167)
177 commits (total: 30,606)
0 new public libcurl function (total: 91)
1 new curl_easy_setopt() option (total: 303)
4 new curl command line option (total: 255)
55 contributors, 34 new (total: 2,922)
35 authors, 20 new (total: 1,170)
1 security fixes (total: 146)

Release presentation

Security

fopen race condition (medium)

CVE-2023-32001. libcurl can be told to save cookies, HSTS and/or alt-svc data to files. When doing this, it called stat() followed by fopen() in a way that made it vulnerable to a TOCTOU (Time of Check, Time of Use) race condition problem.

By exploiting this flaw, an attacker could trick the victim to create or overwrite protected files holding this data in ways it was not intended to.

Changes

curl: add –ca-native and –proxy-ca-native

The command line tool (and library) got new options to ask it to use the systems “native” CA storage. Currently only work on Windows when curl is built to use an OpenSSL fork.

curl: add –trace-ids

This option makes the trace log files include connection and transfer identifiers, which greatly helps debugging transfers doing many (parallel) transfers.

CURLOPT_MAIL_RCPT_ALLOWFAILS replaces CURLOPT_MAIL_RCPT_ALLLOWFAILS

Provide the option without the typo!

add –haproxy-clientip flag to set client IPs

Now users of the tool (and library) pass on specific IP addresses instead of simply using the current one.

add CURLINFO_CONN_ID and CURLINFO_XFER_ID

Two options that allows the application to extract the connection and transfer “Id” of the current transfer, presumably from a debugfunction callback and the likes.

Bugfixes

We have again fixed more than a hundred problems in this release cycle. Here follows a subset that I suspect might be among the most interesting ones.

examples: we’ve added and extended numerous

The ambition is to gradually over time provide examples that show use of all curl_easy_setopt options. We are still way off from that.

http2: numerous smaller and larger fixes

Several regressions and cleanups have been done that improves how HTTP/2 works compared to previous releases.

http2: send HEADER and DATA together

When sending POST requests, libcurl now does a better job in putting the initial outgoing HEADER and DATA frames together, most likely in the same TLS frame.

http3: upload EAGAIN handling

EAGAIN handling for HTTP/3 uploads was fixed, like it was for HTTP/2 as well.

http: fix the outgoing Cookie: header length check

The check that would prevent too long outgoing cookie headers was off by up to a few hundred bytes.

libssh2: use custom memory functions (again)

Bring back use of custom memory functions with libssh2 as otherwise it actually cannot be used with a debug build of curl (or when libssh2 is used as a DLL on windows) due to naive presumptions in the libssh2 API.

runtests: many improvements, leading to -j

Introducing parallel tests.

sectransp: fix EOF handling

A regression caused curl misbehave on end of connection using TLS when built to use Secure Transport.

timeval: use CLOCK_MONOTONIC_RAW if available

For platforms with this clock option, curl now prefers that in an effort to avoid a time that can go backwards.

tool_writeout_json: fix encoding of control characters

The output of control codes in the generated JSON with --json now works better.

urlapi: have *set(PATH) prepend a slash if one is missing

Setting a path using the URL API without a leading slash would previously generate a broken URL when it was extracted. Starting now, libcurl will prepend a slash if there is none.

urlapi: scheme must start with alpha

The URL parser would previously allow a few other characters to start a scheme as well. No more.

tool_parsecfg: accept line lengths up to 10M

The config file parser now allows lines to be up to 10 megabytes. For those odd users generating files with huge data components embedded.

This is the second follow-up patch release in the 8.1.x series due to regressions and bugs that are too annoying to leave lingering around.

Release video

Numbers

the 219th release
0 changes
7 days (total: 9,202)
14 bug-fixes (total: 9,045)
22 commits (total: 30,429
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 302)
0 new curl command line option (total: 251)
13 contributors, 3 new (total: 2,888)
5 authors, 2 new (total: 1,150)
0 security fixes (total: 145)

Bugfixes

configure: quote the assignments for run-compiler

A regression introduced in the previous release made configure fail if the $CC shell variable was set to something else than just a single command name. This now quotes the variable correctly.

configure: without pkg-config and no custom path, use -lnghttp2

Installations without pkg-config where nghttp2 is installed in a default directory would get a link error in the build.

http2: fix EOF handling on uploads with auth negotiation

This was a regression when using HTTP/2 for doing multi-phase authentication methods with POST, like for example Digest.

http3: send EOF indicator early as possible

By better tracking the amount of upload data, curl can avoid a superfluous final zero-length DATA packet and instead send the EOF sooner.

libcurl.m4: remove trailing ‘dnl’ that causes this to break autoconf

The configure macro we ship for other projects to use to detect installed libcurl version now works better.

libssh: when keyboard-interactive auth fails, try password

When a SSH server allows multiple auth methods, and curl tried keyboard-interactive it would wrongly skip trying the password method – if built to use libssh. This bug has been present all since libssh support shipped.