Numbers

the 255th and 256th releases
5 changes
56 days (total: 9,504)
162 bug-fixes (total: 10,050)
246 commits (total: 31,931)
0 new public libcurl function (total: 93)
0 new curl_easy_setopt() option (total: 304)
0 new curl command line option (total: 258)
92 contributors, 56 new (total: 3,133)
37 authors, 15 new (total: 1,252)
4 security fixes (total: 155)

Versions

I first released 8.7.0, but immediately someone pointed out that one of the files in the tarballs was broken, so I fixed the issue, created a new set of tarballs, bumped the version and uploaded the new set. The new release is 8.7.1 but of course it has the same set of changes. We just pretend we did not upload 8.7.0.

Release presentation

Security

• CVE-2024-2004: Usage of disabled protocol. (low)
  When a protocol selection parameter option disables all protocols without adding any then the default set of protocols would remain in the allowed set due to an error in the logic for removing protocols.
• CVE-2024-2398: HTTP/2 push headers memory-leak. (medium)
  When an application tells libcurl it wants to allow HTTP/2 server push, and the amount of received headers for the push surpasses the maximum allowed limit (1000), libcurl aborts the server push. When aborting, libcurl inadvertently does not free all the previously allocated headers and instead leaks the memory.
• CVE-2024-2379: QUIC certificate check bypass with wolfSSL. (low)
  libcurl skips the certificate verification for a QUIC connection under certain conditions, when built to use wolfSSL. If told to use an unknown/bad cipher or curve, the error path accidentally skips the verification and returns OK, thus ignoring any certificate problems.
• CVE-2024-2466: TLS certificate check bypass with mbedTLS. (medium)
  libcurl did not check the server certificate of TLS connections done to a host specified as an IP address, when built to use mbedTLS.

Changes

• configure: add –disable-docs flag. This skips the step generating the manpages, which for many people is unnecessary.
• CURLINFO_USED_PROXY: return bool whether the proxy was used. Useful when having a filter that only lets some transfers use the proxy.
• write-out: add ‘%{proxy_used}’. The same as above but for the tool.
• digest: support SHA-512/256. Support more modern digest authentication.
• DoH: add trace configuration. Now you get more DoH tracing/logging using the general trace mechanism.

Bugfixes

Some of the bugfixes from this cycle that might be worth noticing:

• configure: find libpsl with pkg-config. Makes configure better at finding libpsl and making use of the correct flags and sub-dependencies when linking with it.
• configure: find rustls with pkg-config. Similar adjustment but for rustls.
• cookie: if psl fails, reject the cookie. A run-time failure should not allow the cookie through.
• curl: exit on config file parser errors. We can insist on the config file to be correct as otherwise something unintended might go through.
• curl: make –libcurl output better CURLOPT_*SSLVERSION. This option takes a bitmask made out of two separate enum ranges.
• file: use xfer buf for file:// transfers. The main effect being that it can use a larger buffer which can make faster transfers.
• http: better error message for HTTP/1.x response without status line
• https-proxy: use IP address and cert with ip in alt names. Connecting to a HTTPS proxy using an IP address with a URL also using an IP address and those addresses were different versions, curl would get it wrong.
• mprintf: fix format prefix I32/I64 for windows compilers
• OpenSSL QUIC: adapt to v3.3.x. Pending improvements in OpenSSL is going to enhance curl’s ability to do HTTP/3 using it.
• paramhlp: fix CRLF-stripping files with “-d @file”. curl would do wrong for line ending consisting of CR only
• rustls: make curl compile with 0.12.0. Adjusted to use the modified APIs.
• schannel: fix hang on unexpected server close
• sendf: ignore response body to HEAD. A regression made curl complain if a HEAD request would get body data.
• smtp: fix STARTTLS. Another regression fixed.
• strtoofft: fix the overflow check. The previous overflow check was relying on undefined behavior. This is in code only for platforms without a proper native parser for 64 bit sized numbers.
• TLS: start shutdown only when peer did not already close.
• curl: only parse etag + content-disposition for 2xx.
• curl: accept a blank -w “”
• curl: handle non-existing (out of range) short-options
• curl: change precedence of server Retry-After time
• curl: shorter –help texts. With some polish to make the output look nicer, in particular “curl –help all”.
• transfer.c: break receive loop in speed limited transfers, To make libcurl adapt more precisely to the network speed limit set by the application.

Numbers

the 254th release
7 changes
56 days (total: 9,448)
154 bug-fixes (total: 9,888)
257 commits (total: 31,684)
0 new public libcurl function (total: 93)
1 new curl_easy_setopt() option (total: 304)
0 new curl command line option (total: 258)
65 contributors, 40 new (total: 3,078)
36 authors, 18 new (total: 1,237)
1 security fix (total: 151)

Release presentation

Security

CVE-2024-0853: OCSP verification bypass with TLS session reuse. curl inadvertently kept the SSL session ID for connections in its cache even when the verify status (OCSP stapling) test failed. A subsequent transfer to the same hostname could then succeed if the session ID cache was still fresh, which then skipped the verify status check.

Changes

• Markdown documentation. Most of the libcurl and command line documentation is now written using (basic) markdown instead of previous formats. Easier to read, easier to write.

• CURLE_TOO_LARGE. A new libcurl error code for when “something” is growing too big to be allowed. Like a URL, a HTTP request or similar. Previously it would return out of memory for those situations which caused confusion to users.

• CURLINFO_QUEUE_TIME_T. Applications can now ask libcurl how long a transfer was “queued” internally before it actually started.

• CURLOPT_SERVER_RESPONSE_TIMEOUT_MS. A new millisecond version of the already existing option to allow applications higher resolution control.

• Use GetAddrInfoExW on >= Windows 8. On current Windows versions libcurl will now do asynchronous name resolving by default without using threads, which should be less resource heavy.

• libpsl detection failure in configure causes error. If configure cannot find libpsl it will require the user to say that it should not be used, or to fix the problem. To make people who build curl more aware of the PSL state of the build.

• runtests supports -gl, When you invoke individual test cases on macOS, you can now ask to run it with lldb with -gl just as you have been able to run it with gdb using -g for decades. Helps debugging difficult cases.

Bugfixes

Here some of my favorite bugfixes from this cycle:

configure: add libngtcp2_crypto_boringssl detection. Previously it would only detect and build out of the box with the quictls version of ngtcp2 builds.

configure: when enabling QUIC, check that TLS supports QUIC. More efforts trying to detect wrong and invalid build combinations earlier, to avoid users ending up with broken builds.

all libcurl man page examples are verified in CI. Every man page example now compiles cleanly. This step made us detect and fix numerous tiny mistakes of the most annoying kind: when you copy code from docs and it does not work.

curl shows ipfs and ipns as supported “protocols”. In the regular --version output. Even if they are converted to https:// internally.

curl bsearches command line options. The command line parsing is now magnitudes faster. Of course it will not really be noticeable outside of the most extreme cases.

curl stopped supporting @filename style for --cookie. This syntax was never documented and was not used in any test case. It was risking to cause unwanted surprises.

curl –remove-on-error only removes “real” files. Mostly as a precaution for when users are unclever enough to run curl with elevated privileges and would save to a device or named pipe etc.

curl no longer sets the file comment on Amiga. It would truncate the URL weirdly and also risk leak credentials if such were used in the URL.

lib: reduced use of the download buffer all over. The download buffer has over time been abused for all kinds of buffer purposes. This cycle we have made a lot of such buffer use instead start use their own buffers. With a little luck, this will make us possible use a single download buffer for all transfer in a multi handle, thereby drastically reducing the amount of used memory when doing parallel transfers. With no behavior difference or performance degradation. Details on this will follow later.

lib: use memdup0 instead of malloc + memcpy. This was a common code pattern, and with this we reduce the number of mallocs and memcpys at the same time – which we think is good since they are known “problem functions” that are easy to mess up.

lib: various conversions from malloc to dynbuf. In similar spirit as the above, we continued to switch more functions away from using malloc and family to instead use the internal dynbuf API for managing dynamic buffers in a way that is less likely to cause memory related issues.

resolving: with modern c-ares, use its default timeout. It means tighter timeouts by default but also that this combo now also respect the timeout that can be specified in resolve.conf.

headers API: make sure the trailing newline is never stored. A header with no content on the right side of the colon would erroneously get its trailing newline stored as content

mprintf: overhaul, performance and bugfixes. Now the curl printf functions work even more similar to the glibc counterparts especially when provided illegal %-combinations and when using the <num>$ operator. Performance measurements on this new code also says this code now executes around 30% faster on commonly used format strings.

ftp: handle the PORT parsing without allocation. Minor cleanup.

http3: initial support for OpenSSL 3.2 QUIC stack. The forth QUIC backend in curl is here.

http: check for “Host:” case insensitively. If you would ask to disable this header with a different casing that what was compared, it would still send an empty header in the request.

http: only act on 101 responses when they are HTTP/1.1. If a HTTP response says another protocol version with a 101 response, it is now considered an illegal combination.

openldap: fix an LDAP crash. LDAP without TLS would crash on basic use.

openldap: fix STARTTLS. It was recently broken in a refactor.

Next

There are no revolutionary changes in the pipe, but there are a series of things we most likely are going to land in the next cycle making the next version number likely to become 8.7.0.

Release presentation

Numbers

the 253rd release
2 changes
56 days (total: 9,392)
188 bug-fixes (total: 9,734)
266 commits (total: 31,427)
0 new public libcurl function (total: 93)
0 new curl_easy_setopt() option (total: 303)
0 new curl command line option (total: 258)
78 contributors, 43 new (total: 3,039)
40 authors, 19 new (total: 1,219)
2 security fixes (total: 150)

Security

cookie mixed case PSL bypass

(CVE-2023-46218) This flaw allows a malicious HTTP server to set “super cookies” in curl that are then passed back to more origins than what is otherwise allowed or possible. This allows a site to set cookies that then would get sent to different and unrelated sites and domains.

It could do this by exploiting a mixed case flaw in curl’s function that verifies a given cookie domain against the Public Suffix List (PSL). For example a cookie could be set with domain=co.UK when the URL used a lower case hostname curl.co.uk, even though co.uk is listed as a PSL domain.

HSTS long file name clears contents

(CVE-2023-46219) When saving HSTS data to an excessively long file name, curl could end up removing all contents, making subsequent requests using that file unaware of the HSTS status they should otherwise use.

Changes

We have only logged two changes for this cycle.

gnutls supports CURLSSLOPT_NATIVE_CA

If you use libcurl built to use GnuTLS, you too can now set this bit and get to use the Windows CA store directly from within libcurl instead of having to provide a separate PEM file. When your application runs on Windows. libcurl already previously support this for OpenSSL and wolfSSL.

See the CURLOPT_SSL_OPTIONS documentation for details.

HTTP3 with ngtcp2 no longer experimental

The first HTTP/3 code in curl was merged into git in 2019. Now we ship HTTP/3 support non-experimental for the first time. curl supports HTTP/3 using several different backends but this news is only for HTTP/3 built to use ngtcp2 + nghttp3. The other HTTP/3 backends remain experimental for the time being.

Note that in order to be able to use ngtcp2 you need to build with a TLS library that offers the necessary API. This means you need to use one of the following libraries: wolfSSL, quictls, BoringSSL, libressl, AWS-LC or GnuTLS. Let me stress that OpenSSL is not in that list.

Bugfixes

As usual, here I have collected a few of my favorite fixes from this cycle.

improved IPFS and IPNS URL support

Turned out there were some IPFS URLs that curl did not manage properly.

doh: use PIPEWAIT when HTTP/2 is attempted

This makes curl prefer doing DoH requests multiplexed over a single connection rather than doing them as two separate connections. Most DoH lookups do one request for an IPv4 response and a separate one for IPv6.

duphandle: several OOM cleanups

If libcurl runs out of memory in the middle of the curl_easy_duphandle function, it could previously do several mistakes, including free-twice.

hostip: show the list of IPs when resolving is done

The verbose mode now shows the full list of IP addresses that was resolved from the name, before it continues to try to connect to them in a serial fashion.

aws-sigv4: canonicalize valueless query params

Turns out there was yet another glitch in the sigv4 logic that made curl send the wrong checksum for URLs using “valueless” query parameters.

hyper: temporarily remove HTTP/2 support

The hyper integration for HTTP/2 was incorrect and has therefore been disabled for now. hyper support in curl is still experimental.

drop vc10, vc11 and vc12 projects from dist, add vc14.20

We generate less project files for older Visual Studio versions but we added one for a recent version. Going forward, we might soon drop them completely from the tarballs since they can now be generated with cmake.

openssl: include SIG and KEM algorithms in verbose

curl now presents more details from the TLS handshake in the verbose output when using OpenSSL.

openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs

The keyword here is intermediate. Previously, curl would ignore those which would lead to handshake errors because that is not what users expect when you use the native CA store.

openssl: when a session-ID is reused, skip OCSP stapling

When trying to do verify status, the more technical name for OCSP stapling, after a connection has been established using a session-ID, it would return error.

make SOCKS5 use the CURLOPT_IPRESOLVE choice

There are just too many combinations, but we find it likely that a user that asks for a specific IP version probably also wants it for the traffic over the SOCKS proxy.

tool: support bold headers in Windows

The feature that has been provided on other systems since 2018 now comes to Windows.

make the carriage return fit wide progress bars

The progress bar set with -#/--progress-bar had its math off by one, which made it not output the carriage return character when the terminal width was wider than 256 columns, making the output wrong.

url: find scheme with a “perfect hash”

The internal function that scans the list of supported URL schemes no longer iterates through the list but instead uses a “perfect hash”. Although much faster, that was hardly a function that caused performance problems before so it is not likely to actually be measurable.

use ALPN “http/1.1” for HTTP/1.x, including HTTP/1.0

To interoperate better with legacy servers, curl now sends http/1.1 in the ALPN field even when it wants to speak HTTP/1.0. This, because http/1.1 and h2 were the original ALPN codes and some of the old servers that support ALPN from those days don’t know about http/1.0 for ALPN.

all libcurl man pages examples compile cleanly

All (almost five hundred) libcurl man pages have EXAMPLE sections showing their use. Starting now, all of those sections are test-compiled as part of the CI builds to catch mistakes.

Welcome to this new curl release!

Release video

Numbers

the 251st release
9 changes
49 days (total: 9,308)
174 bug-fixes (total: 9,415)
296 commits (total: 30,942)
1 new public libcurl function (total: 92)
0 new curl_easy_setopt() option (total: 303)
2 new curl command line option (total: 257)
80 contributors, 50 new (total: 2,977)
40 authors, 20 new (total: 1,193)
1 security fix (total: 146)

Numbers notes:

1. the release counter now also includes project releases done before the name was changed to curl.
2. The number of security fixes is adjusted due to the recently rejected CVE-2023-32001

Security

We publish a security advisory in association with today’s release.

HTTP headers eat all memory

[CVE-2023-38039] When curl retrieves an HTTP response, it stores the incoming headers so that they can be accessed later via the libcurl headers API.

However, curl did not have a limit in how many or how large headers it would accept in a response, allowing a malicious server to stream an endless series of headers and eventually cause curl to run out of heap memory.

Changes

curl: make %output{} in -w specify a file to write to

The super handy option –write-out become even more convenient now as it can redirect its output into a specific file and not just stdout and stderr.

curl: add “variable” support

The new variable concept now only lets users use environment variables on config files but also opens up for new ways to use curl command lines effectively.

remove gskit support

The gskit TLS library is no longer a provided option when building curl.

remove NSS support

The NSS TLS library is no longer a provided option when building curl. curl still supports building with twelve different TLS libraries even after the removal of these two.

configure –disable-bindlocal builds curl without local binding support

As a next step in the gradual movement to allow more and more features to get enabled/disabled at build time, the time came to the bindlocal function, which is the feature that binds the local end of a connection. Primarily intended for tiny-curl purposes when you aim for a minimal footprint build.

make tracing available in non-debug builds

Starting now, libcurl offers curl_global_trace and curl offers –trace-config to ask for what specific details to include in the verbose logging output. This is a way for a non-debug build to provide more protocol level details from transfers in ways that were previously not possible. Allows for users to report bugs better and provide more insights from real-world problematic scenarios.

CURLOPT_MAXREDIRS defaults to 30

As a precaution, we change the default from unlimited to 30.

CURLU_PUNY2IDN – convert punycode to IDN

The URL API gets the ability to convert to an International Domain Name when given a punycode version. Previously it could only do the conversion in the other direction.

wolfssl: support loading system CA certificates

curl built with wolfSSL now can use the “native CA” option which then makes it possible to use the native CA store on several platforms instead of using a separately provided external file.

Bugfixes

More than 160 bugfixes are logged for this release, but here are a few selected highlights.

accept and parse IPv6 addresses in alt-svc response headers

Previously curl would not parse and accept such hosts.

c-ares: reduce timeout to 2000ms

The default c-ares DNS timeout is set to the same time that c-ares itself has changed to in their next pending release.

make CURLOPT_HAPROXY_CLIENT_IP set the source IP

It was wrongly set as destination instead of source.

cmake: ten separate improvements

Numerous smaller and larger fixes that made the cmake build of curl several notches better.

stop halving the remaining connect timeout when less than 600 ms left

When curl connects to a host that resolves to multiple IP addresses, it allows half the timeout time for the current IP before it moves on to attempt the next IP in the list. That “halving” is now stopped when there is less than 600 milliseconds left to reduce problems with too short times.

docs: rewrite to present tense

Most of the curl documentation now says “this option does this” instead of “this option will do this”

escape all dashes (ASCII minus) to avoid Unicode hyphens in curl.1 man page

It turns out the curl man page as generated previously, would make the man command use a Unicode hyphen instead of ASCII minus when displayed. This broke copy and paste and it made it impossible to properly search for minus/dash when viewing the man page.

accept leading whitespace on first HTTP response header

curl is now less strict if the first HTTP/1 response header starts with space or tab, thus looking like it is a “fold” when it not. Other commonly used tools/browsers accept this kind of bad syntax and so does curl now.

avoid too early HTTP/2 connection re-use/multiplexing

When doing lots of parallel transfers curl might need to create a second connection when the first reaches its maximum number of streams. In that situation, curl would try to multiplex on that new connection too early, already before it was properly setup and be ready for use, leading to transfer errors.

http/http2/http3: fix sending large requests

Logic for all supported HTTP versions had (different) issues in handling sending very large requests.

aws-sigv4: canonicalize the query

Using aws-sigv4 authentication would fail if the query part was not manually crafted to be correct: sorted, uppercase %-encoding and all the name/value pairs alpha-sorted. Now curl does this itself.

make aws-sigv4 not require TLS to be used

The –aws-sigv4 option no longer requires an HTTPS:// URL to be used.

lib: move mimepost data from ->req.p.http to ->state

The moving of internal data from one struct to another made data survive between two requests and thus fixed a bug involving redirects with MIMEPOST that needed to rewind.

use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set

Turns out curl would still resolve both IPv4 and IPv6 names even if ipv6-only connections were being requested, thus getting some extra names in vein.

system.h: add CURL_OFF_T definitions on HP-UX with HP aCC

Starting now, curl builds properly on more HP-UX machines.

tests: update cookie expiry dates to far in the future

curl’s test suite now runs fine even when executed in a year after 2038.

tool_filetime: make -z work with file dates before 1970

The -z option can get the file date off a local file and use that in a HTTP time condition request, but if the file was older than January 1 1970 it would act wrongly.

transfer: also stop the sending on closed connection

When curl sent off a HTTP/1 request and the connection was closed before the sending was complete, curl could end up not detecting that and ending the transfer correctly.

don’t set TIMER_STARTTRANSFER on first send

Adjustments were made to make this timestamp work as actually documented.

make zoneid duplicated in curl_url_dup

This dup function did not correctly duplicate the zone id from the source handle, making it an incomplete duplicate.

quic: don’t set SNI if hostname is an IP address

curl would wrongly populate the SNI field with the IP address when doing QUIC connections to such.

Next

This is a dot-zero release. If there are any important enough regressions shipped in this version, we will do a follow-up release within shortly. Report all and any problems you spot.

Welcome. Due to some annoying regressions in the previous release we think we owned it to everyone to do a quick patch follow-up.

Download curl 8.2.1 from https://curl.se.

Release presentation

Numbers

the 221st release
0 changes
7 days (total: 9,259)
27 bug-fixes (total: 9,194)
37 commits (total: 30,646)
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 303)
0 new curl command line option (total: 255)
20 contributors, 7 new (total: 2,927)
13 authors, 3 new (total: 1,173)
0 security fixes (total: 146)

Bugfixes

Here are some the most important fixes in this release

configure: check for nghttp2_session_get_stream_local_window_size

We use this function now, introduced in nghttp2 1.15.0, released in September 2016.

return IPv6 first for localhost resolves

Resolving “localhost” did not return the (fixed) addresses in the correct order. It now returns IPv6 as the first.

http2 regression on upload EOF handling

When we added an optimization in the previous release we missed a code path that sometimes lead to “hanging” uploads over HTTP/2.

os400: correct EXPECTED_STRING_LASTZEROTERMINATED

curl builds fine for “IBM i” again.

quiche: fix lookup of transfer at multi

Doing multiplexed HTTP/3 over multiple connections with quiche works much better.

mkhelp: strip off escape sequences

The command sequence that generates the man page display code for the --manual option did at some point regress to include escape sequences. Now those sequences are properly filtered out.

fix build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T

A build problem was fixed for these rare platforms.

do not clear the credentials on redirect to absolute URL

Yet another regression that we allowed because we apparently did not have a test for this! Now we have a test and redirects to the same origin when using -u for credentials now send the credentials even in the redirected request.

Welcome to another curl release. You know how this dance goes…

Numbers

the 220th release
5 changes
50 days (total: 9,252)
122 bug-fixes (total: 9,167)
177 commits (total: 30,606)
0 new public libcurl function (total: 91)
1 new curl_easy_setopt() option (total: 303)
4 new curl command line option (total: 255)
55 contributors, 34 new (total: 2,922)
35 authors, 20 new (total: 1,170)
1 security fixes (total: 146)

Release presentation

Security

fopen race condition (medium)

CVE-2023-32001. libcurl can be told to save cookies, HSTS and/or alt-svc data to files. When doing this, it called stat() followed by fopen() in a way that made it vulnerable to a TOCTOU (Time of Check, Time of Use) race condition problem.

By exploiting this flaw, an attacker could trick the victim to create or overwrite protected files holding this data in ways it was not intended to.

Changes

curl: add –ca-native and –proxy-ca-native

The command line tool (and library) got new options to ask it to use the systems “native” CA storage. Currently only work on Windows when curl is built to use an OpenSSL fork.

curl: add –trace-ids

This option makes the trace log files include connection and transfer identifiers, which greatly helps debugging transfers doing many (parallel) transfers.

CURLOPT_MAIL_RCPT_ALLOWFAILS replaces CURLOPT_MAIL_RCPT_ALLLOWFAILS

Provide the option without the typo!

add –haproxy-clientip flag to set client IPs

Now users of the tool (and library) pass on specific IP addresses instead of simply using the current one.

add CURLINFO_CONN_ID and CURLINFO_XFER_ID

Two options that allows the application to extract the connection and transfer “Id” of the current transfer, presumably from a debugfunction callback and the likes.

Bugfixes

We have again fixed more than a hundred problems in this release cycle. Here follows a subset that I suspect might be among the most interesting ones.

examples: we’ve added and extended numerous

The ambition is to gradually over time provide examples that show use of all curl_easy_setopt options. We are still way off from that.

http2: numerous smaller and larger fixes

Several regressions and cleanups have been done that improves how HTTP/2 works compared to previous releases.

http2: send HEADER and DATA together

When sending POST requests, libcurl now does a better job in putting the initial outgoing HEADER and DATA frames together, most likely in the same TLS frame.

http3: upload EAGAIN handling

EAGAIN handling for HTTP/3 uploads was fixed, like it was for HTTP/2 as well.

http: fix the outgoing Cookie: header length check

The check that would prevent too long outgoing cookie headers was off by up to a few hundred bytes.

libssh2: use custom memory functions (again)

Bring back use of custom memory functions with libssh2 as otherwise it actually cannot be used with a debug build of curl (or when libssh2 is used as a DLL on windows) due to naive presumptions in the libssh2 API.

runtests: many improvements, leading to -j

Introducing parallel tests.

sectransp: fix EOF handling

A regression caused curl misbehave on end of connection using TLS when built to use Secure Transport.

timeval: use CLOCK_MONOTONIC_RAW if available

For platforms with this clock option, curl now prefers that in an effort to avoid a time that can go backwards.

tool_writeout_json: fix encoding of control characters

The output of control codes in the generated JSON with --json now works better.

urlapi: have *set(PATH) prepend a slash if one is missing

Setting a path using the URL API without a leading slash would previously generate a broken URL when it was extracted. Starting now, libcurl will prepend a slash if there is none.

urlapi: scheme must start with alpha

The URL parser would previously allow a few other characters to start a scheme as well. No more.

tool_parsecfg: accept line lengths up to 10M

The config file parser now allows lines to be up to 10 megabytes. For those odd users generating files with huge data components embedded.

This is the second follow-up patch release in the 8.1.x series due to regressions and bugs that are too annoying to leave lingering around.

Release video

Numbers

the 219th release
0 changes
7 days (total: 9,202)
14 bug-fixes (total: 9,045)
22 commits (total: 30,429
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 302)
0 new curl command line option (total: 251)
13 contributors, 3 new (total: 2,888)
5 authors, 2 new (total: 1,150)
0 security fixes (total: 145)

Bugfixes

configure: quote the assignments for run-compiler

A regression introduced in the previous release made configure fail if the $CC shell variable was set to something else than just a single command name. This now quotes the variable correctly.

configure: without pkg-config and no custom path, use -lnghttp2

Installations without pkg-config where nghttp2 is installed in a default directory would get a link error in the build.

http2: fix EOF handling on uploads with auth negotiation

This was a regression when using HTTP/2 for doing multi-phase authentication methods with POST, like for example Digest.

http3: send EOF indicator early as possible

By better tracking the amount of upload data, curl can avoid a superfluous final zero-length DATA packet and instead send the EOF sooner.

libcurl.m4: remove trailing ‘dnl’ that causes this to break autoconf

The configure macro we ship for other projects to use to detect installed libcurl version now works better.

libssh: when keyboard-interactive auth fails, try password

When a SSH server allows multiple auth methods, and curl tried keyboard-interactive it would wrongly skip trying the password method – if built to use libssh. This bug has been present all since libssh support shipped.

Only 6 days since the previous release we are again here with a curl release. It turned out 8.1.0 had some rather nasty regressions that we felt were urgent enough to warrant another round on the dance floor. So here goes curl 8.1.1. A bugfix release.

Release presentation

Numbers

the 218th release
0 changes
6 days (total: 9,195)
25 bug-fixes (total: 9,031)
40 commits (total: 30,407
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 302)
0 new curl command line option (total: 251)
19 contributors, 10 new (total: 2,885)
13 authors, 6 new (total: 1,148)
0 security fixes (total: 145)

Bugfixes

Some of the highlights of this release include…

cmake: avoid list(PREPEND)

This use of a too new cmake feature snuck itself into the build in the last release which caused trouble for people using older cmake versions.

cmake: repair cross compiling

A recently added cmake check did not have the correct precautions added for cross-compiling which broke such builds.

configure: generate a script to run the compiler

The configure script has an elaborate check that verifies provided if libraries can be used at run-time. This turned out complicated when the compiler itself uses libraries that configure checks for by setting the LD_LIBRARY_PATH since that path also affects the compiler!

http2: double http request parser max line length

The last word is probably not said about this logic, but capping the max request header line size to 4KB was too narrow and caused application breakages. Now the limit is at 8KB.

http2: increase stream window size to 10 MB

It turned out that even though we have a flexible HTTP/2 window concept, download performance could suffer and now we have bumped the window size again significantly.

http2: upload improvements

In particular doing uploads that are aborted prematurely by a reset when for example a 404 is returned before the entire upload was done could cause issues.

rename struct ‘http_req’ to ‘httpreq’

The development branch of FreeBSD (14) introduced a struct in one of the public headers that name-collided an internal struct libcurl uses. The bug exists in FreeBSD’s header, but we renamed ours anyway to work around the problem while the FreeBSD team fixes their end.

better error message when URLs fail to parse

Since we have a fairly elaborate identification of exactly what fails when the URL parser rejects a URL, this now helps users to better understand what curl does not like.

urlapi: allow numerical parts in the host name

The URL parser was far too strict in rejecting host names because they were “invalid IPv4” when in fact they should be treated as host names instead. Probably the worst regression added in 8.1.0. In fact, the URL parser basically cannot refuse a host name for not being a valid IPv4 since then it can get passed through to the name resolver which can then still find it in /etc/hosts etc.

We are back with the first release since that crazy March day when we did two releases on the same day. First 8.0.0 shipped that bumped the major version for the first time in decades. Then curl 8.0.1 followed just hours after, due to a serious mess-up in the factory lines.

Release video presentation

Numbers

the 217th release
3 changes
58 days (total: 9,189)
185 bug-fixes (total: 9,006)
322 commits (total: 30,367
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 302)
1 new curl command line option (total: 251)
64 contributors, 35 new (total: 2,875)
37 authors, 17 new (total: 1,142)
4 security fixes (total: 145)

Security

We disclose four new curl security vulnerabilities today, three of them at severity Low and one of them at Medium. This also means that 3,840 USD was awarded as bug bounties in this release cycle.

UAF in SSH sha256 fingerprint check

[CVE-2023-28319] libcurl offers a feature to verify an SSH server’s public key using a SHA 256 hash. When this check fails, libcurl would free the memory for the fingerprint before it returns an error message containing the (now freed) hash.

siglongjmp race condition

[CVE-2023-28320] libcurl provides several different backends for resolving host names, selected at build time. If it is built to use the synchronous resolver, it allows name resolves to time-out slow operations using alarm() and siglongjmp().

When doing this, libcurl used a global buffer that was not mutex protected and a multi-threaded application might therefore crash or otherwise misbehave.

IDN wildcard match

[CVE-2023-28321] curl supports matching of wildcard patterns when listed as “Subject Alternative Name” in TLS server certificates. curl can be built to use its own name matching function for TLS rather than one provided by a TLS library. This private wildcard matching function would match IDN (International Domain Name) hosts incorrectly and could as a result accept patterns that otherwise should mismatch.

more POST-after-PUT confusion

[CVE-2023-28322] When doing HTTP(S) transfers, libcurl might erroneously use the read callback (CURLOPT_READFUNCTION) to ask for data to send, even when the CURLOPT_POSTFIELDS option has been set, if the same handle previously was used to issue a PUT request which used that callback.

This flaw may surprise the application and cause it to misbehave and either send off the wrong data or use memory after free or similar in the second transfer.

Changes

This release has only three real changes. One bigger and two smaller:

HTTP/2 over proxy

libcurl can now negotiate and use HTTP/2 when it is told to use a HTTPS proxy (details in the CURLOPT_PROXYTYPE man page), and the command line tool can of course switch it on using the --proxy-http2 option. Explained more in this blog post.

refuse to resolve the .onion TLD

When a host name ending with .onion is passed on to the name resolver functions, they will cause an error and will not be resolved. Like RFC 7686 tells us.

curl’s -w option can now output URL components

The list of variables was extended by a whole range of new ones. Possibly best learned by checking out the writeout section in everything curl.

Bugfixes

The official counter says we did more than 180 bugfixes in his release cycle. Here follows some of my favorites:

checksrc fixes

We made it better at checking the code style for three distinct code situations – and then updated the source code accordingly.

cmake fixes

• bring in the network library on Haiku
• do not add zlib headers for OpenSSL
• make config version 8 compatible with 7
• set SONAME for SunOS too

only do transfer-encoding compression if asked to

Transfer encodings other than “chunked” are rarely used. Up until now libcurl would still activate automatic decompression if such was used, even if it was not asked for by the application.

bring back support for SFTP path ending in /~

A regression made a URL that ends with /~ no longer make a directory listing because the URL does not end with a slash. Now we bring back that behavior, even if it goes a little against the standard behavior.

never allocate dynbufs larger than “too big”

The general dynamic buffer system no longer allocates more memory than what the specific buffer is allowed to grow to. An optimization.

various gskit compile errors in OS400

Makes curl build fine there again.

enforce a maximum DNS cache size independent of timeout value

The DNS cache entries are purged on age only (default 60 seconds). With this new code, libcurl limits caps the maximum total amount of DNS cache entries to 30,000.

libssh2: fix crash in keyboard callback

Better SCP and SFTP when built with libssh2.

libssh: tell it to use SFTP non-blocking

Better SCP and SFTP when built with libssh.

add multi-ignore logic to multi_socket_action

The improved signal ignore logic for curl_multi_perform in 8.0.0 is now also done for curl_multi_socket_action. For better performance.

remove PENDING + MSGSENT handles from the main linked list

Not yet activated transfers and the transfers that are already completed, are now moved away off the main linked list. For performance.

runtests: prepare for parallel

Lots of cleanups and smaller fixes have been merged during this cycle in preparation for the pending introduction of parallel tests.

verify socketpair with a random value

The custom socketpair implementation used for platforms without a native one, was changed to use truly random values when verifying that the pipe works.

Fix ‘Location:’ formatting for early VTE terminals

The special terminal highlighting of the URL that is shown in the Location: header is now disabled for some terminals that can’t display it properly.

urlapi polish

Several different bugs and improvements were made. Including:

• cleanups and performance improvements
• detect and error on illegal IPv4 addresses
• prevent setting invalid schemes
• URL encoding for the URL missed the fragment

enhanced WebSocket en-/decoding

Parts of the websocket parser code was rewritten to fix bugs.