Welcome to the final and last release in the series seven. The next release is planned and intended to become version 8.

Numbers

the 213th release
5 changes
56 days (total: 9,098)
173 bug-fixes (total: 8,665)
250 commits (total: 29,821)
0 new public libcurl function (total: 91)
0 new curl_easy_setopt() option (total: 302)
1 new curl command line option (total: 250)
78 contributors, 41 new (total: 2,812)
42 authors, 18 new (total: 1,119)
3 security fixes (total: 135)

Release presentation

Security

This time we bring you three security fixes. All of them covering cases for which we have had problems reported and fixed before, but these are new subtle variations.

• CVE-2023-23914: HSTS ignored on multiple requests

• CVE-2023-23915: HSTS amnesia with –parallel

• CVE-2023-23916: HTTP multi-header compression denial of service

Changes

• Two changes for HTTP/3: CURL_HTTP_VERSION_3ONLY was added for the library and --http3-only was added to the tool.
• Two changes for HSTS: the HSTS cache can now be shared between libcurl handles, and subsequently the curl tool now shares the HSTS between transfers.
• The URL API got the new flag CURLU_PUNYCODE which allows and application to get the punycode version of a host name/URL.
• curl -w now offers %{certs} and %{num_certs} which outputs the server certificate(s).

Bugfixes

While we count over 140 individual bugfixes merged for this release, here follows a curated subset of some of the more interesting ones.

http/3 happy eyeballs

When asking for HTTP/3, curl will now also try older HTTP versions with a slight delay so that if HTTP/3 does not work, it might still succeed with and use an older version.

update all copyright lines and remove year ranges

Mentioned separately.

allow up to 10M buffer size

An application can now set drastically larger download buffers. For high speed/localhost transfers of some protocols this might sometimes make a difference.

curl: output warning at –verbose output for debug-enabled version

To help users realize when they use a debug build of curl, it now outputs a warning at the top of the --verbose output. We strongly discourage users to ship or use such builds in production.

websocket: multiple bugfixes

WebSocket support remains an experimental feature in curl but it is getting better. Several smaller and bigger bugs were squashed. Please continue to try it and report any problems and we can probably consider removing the experimental label soon.

dict: URL decode the entire path always

If you used a DICT URL it would sometimes do wrong as it previously only URL decoded parts of the path when using it. Now it correctly decodes the entire thing.

URL-encode/decode much faster

The libcurl functions for doing these conversions were sped up significantly. In the order of 3x and 7x.

haxproxy: send before TLS handhshake

The haproxy details are now properly sent before the TLS handshake takes place.

HTTP/[23]: continue upload when state.drain is set

Fixes a stalling problem when data is being uploaded and downloaded at the same time.

http2: aggregate small SETTINGS/PRIO/WIN_UPDATE frames

Optimizes outgoing frames for HTTP/2 into doing more in fewer sends.

openssl: store the CA after first send (ClientHello)

By changing the order of things, curl is better off spending CPU cycles while waiting for the server’s response and thereby making the entire handshake process complete faster.

curl: repair –rate

A regression in 7.87.0 made this feature completely broken. Now back on track again.

HTTP/2 much faster multiplexed transfers

By improving the handling of multiple concurrent streams over a single connection, curl now performs such transfers much faster than before. Sometimes an almost 3x speedup.

noproxy: support for space-separated names is deprecated

The parser that parses the “noproxy” string accepts plain space (without comma) as separators, while hardly any other tool or library does. This matters because it can be set in an environment variable. This accepted space-only separation is now marked as deprecated.

nss: implement data_pending method

The NSS backend was improved to work better for cases when the socket has been drained of data and only the NSS internal buffers has it, which could lead to curl getting stalled or losing data. Note: NSS support is marked for removal later in 2023.

socketpair: allow localhost MITM sniffers

curl has an internal socketpair emulation function for Windows. The way it worked did not allow MITM sniffers, but instead return error if such a thing was detected. It turns out too many users run tools on Windows that do this, so we have changed the logic to accept their presence and use.

tests-httpd: infra to run curl against an apache httpd

An entirely new line of tests that opens up new ways to test and verify our HTTP implementations in ways we could not do before. It uses pytest and an apache httpd server with special test modules.

curl: fix hiding of command line secrets

A regression.

curl: fix error code on bad URL

If you would use an invalid URL for upload, curl would erroneously report the problem as “out of memory” which unsurprisingly greatly confused users.

Numbers

the 212th release
5 changes
56 days (total: 9,042)
155 bug-fixes (total: 8,492)
238 commits (total: 29,571)
0 new public libcurl function (total: 91)
2 new curl_easy_setopt() option (total: 302)
1 new curl command line option (total: 249)
83 contributors, 40 new (total: 2,771)
42 authors, 20 new (total: 1,101)
2 security fixes (total: 132)
Bug Bounties total: 48,580 USD

Release presentation

At 10:00 CET (9:00 UTC) on December 21, Daniel live-streams the release presentation on twitch. This paragraph will later be replaced by a link to the YouTube version of that video.

Security

Two security advisories this time around, severity low and medium.

CVE-2022-43551: Another HSTS bypass via IDN

The HSTS logic could be bypassed if the host name in the given URL first uses IDN characters that get replaced to ASCII counterparts as part of the IDN conversion. Like using the character UTF-8 U+3002 (IDEOGRAPHIC FULL STOP) instead of the common ASCII full stop (U+002E). Then in a subsequent request, it does not detect the HSTS state and makes a clear text transfer. Because it would store the info IDN encoded but look for it IDN decoded.

CVE-2022-43552: HTTP Proxy deny use-after-free

When an HTTP PROXY denied to tunnel SMB or TELNET, curl would use a heap-allocated struct after it had been freed in its transfer shutdown code path.

Changes

–url-query

curl’s 249th command line option adds data to the query part of the URL.

CURLOPT_QUICK_EXIT

Tell libcurl to not wait for any DNS threads on exit.

CURL_WRITEFUNC_ERROR

New and easier way to signal write callback errors.

CURLOPT_CA_CACHE_TIMEOUT

libcurl can now cache the CA store in memory, as I blogged about separately.

feature names added to curl_version_info_data

The struct returned by curl_version_info now returns all built-in features listed by name. This is a preparation to allow applications to adapt slowly and get ready for the future moment when the features can no longer fit in in the 32 bit fields previously used for this purpose.

Bugfixes

Better base64

The encoder now allocates the output using a more appropriate size, and both the encoder and decoder implementations are much faster.

hyper

We fixed a few issues in the hyper backend and are down to just 12 remaining disabled tests to address.

gen.pl: fix the linkifier

This script generates the curl.1 man page and make sure to properly mark references correctly, so that the man page can get rendered as we webpage with correct links etc on the website. This time we made it work better and therefore more cross-references in the man page is now linked correctly in the web version.

tool: override the numeric locale and set “C” by force

In previous curl versions it mistakenly used the locale when parsing floating point numbers, which then made the tool hard to use in scripts which would run in multiple locales. An example is the timeout option specified with -m / --max-time as number of seconds with a fraction. Now it requires the decimal separator to always be a dot/period independently of the user’s locale.

tool: timeout in the read callback

The command line tool can now timeout reading data better, for example when using telnet:// with a timeout option and the user does not press any key and nothing happens over the network.

curl_get_line: allow last line without newline char

Because of a somewhat lazy recent fix, the .netrc parsed and other users of the nternal curl_get_line() function would ignore the last line if it did not end with a newline. This is no more.

support growing FTP files with CURLOPT_IGNORE_CONTENT_LENGTH

If this option is set, also known as --ignore-content-length on the command line, curl will not complain if the size grows from the moment the FTP transfer starts until it ends. Thus allowing it to grow while being transferred.

do not send PROXY more than once

The HAproxy protocol line could get sent more than once and thus break stuff.

feature deprecation warnings in gcc

A number of outdated libcurl options and functions are now tagged as deprecated, which will cause compiler warnings when used in application code for users of gcc 6.1 or later. Deprecated here means that we recommend using other, more modern, alternatives.

parse numbers with fixed known base 10

In several places in curl and libcurl source code we would allow numbers to be specified using octal or hexadecimal while decimal was the only expected and documented base. In order to minimize surprises and for consistency, we now limited them as far as possible to only accepting decimal numbers.

rewind BEFORE request instead of AFTER previous

When curl is used to send a request, for example a POST, and there is reason for it to send it again, like if there is a redirect or an ongoing authentication process, it would previously rewind the stream at the end of that transfer first transfer in order to have it done when the next transfer is about to get done. Now, it instead does the rewind first in the second request. This, because there are times when the second request are not done, and the rewind may not work. So, such a failing rewind can be avoided by not doing it until it is strictly necessary.

noproxy

Several independent regressions were fixed – in spite of the new set of test cases added for testing this feature in the previous release. Noproxy is the support for the NO_PROXY environment variable and related options.

openssl: prefix errors with ‘[lib]/[version]:’

To help users understand errors and their origins a little better, libcurl will now prefix error messages originating from OpenSSL (and forks) with the name of the flavor and its version number.

RTSP auth works again

This functionality was broken a few versions back and now it has finally been fixed again.

runtests: –no-debuginfod now disables DEBUGINFOD_URLS

valgrind and gdb support downloading stuff at the moment of need if this environment variable is set. Previously the curl test running script would unset that variable unconditionally, but now it will not and instead offer an option that unsets it – for the cases where the environment variable causes problems (such as performance slowdowns).

HTTP/3 tests

We finally have the first infrastructure merged for doing and running HTTP/3 specific tests in the curl test suite. Now we can better avoid regressions going forward. This is only the beginning and I expect us to expand and grow these tests going forward.

determine the correct fopen option for -D

When saving response headers into a dedicated file with curl’s -D, –dump-header option, curl would be inconsistent about when to create a new file and when to append do it. Now it acts exactly as documented.

better error message for -G with bad URL

Several users figured out curl showed misleading error messages when -G was used in combination with a malformed URL. This is now improved.

repair IDN for proxies

A recent fix we landed for IDN for host names accidentally simultaneously broke it for proxies…

cmake: set the soname on the shared library

Using cmake to build libcurl as a shared library on Linux and several other systems, will now set the SONAME number correctly in the same style and with the same number that the autotools build uses.

WebSocket polish

• fixes for partial frames and buffer updates
• now returns CURLE_NOT_BUILT_IN when websockets support is not built in
• returns error properly when the connection is closed

TLS goes connection filters => more HTTPS-proxy

As a direct result of the internal refactor and introduction of connection filters also for TLS, curl now supports HTTPS-proxy for a wider selection of TLS backends than previously.

Credits

Release image by @sny@mas.to

Welcome to another curl release. You know the drill…

Numbers

the 211th release
2 changes
56 days (total: 8,986)
192 bug-fixes (total: 8,337)
314 commits (total: 29,331)
3 new public libcurl function (total: 91)
1 new curl_easy_setopt() option (total: 300)
0 new curl command line option (total: 248)
74 contributors, 43 new (total: 2,733)
42 authors, 17 new (total: 1,082)
4 security fixes (total: 130)
Bug Bounties total: 46,180 USD

Release presentation

Security

This release contains fixes for four separate security vulnerabilities.

CVE-2022-32221: POST following PUT confusion

When doing HTTP(S) transfers, libcurl might erroneously use the read callback (CURLOPT_READFUNCTION) to ask for data to send, even when the CURLOPT_POSTFIELDS option has been set, if the same handle previously was used to issue a PUT request which used that callback.

This flaw may surprise the application and cause it to misbehave and either send off the wrong data or use memory after free or similar in the subsequent
POST request.

The problem exists in the logic for a reused handle when it is changed from a PUT to a POST.

CVE-2022-35260: netrc parser out-of-bounds access

curl can be told to parse a .netrc file for credentials. If that file ends in a line with consecutive non-white space letters and no newline, curl could read past the end of the stack-based buffer, and if the read works, write a zero byte possibly beyond its boundary.

This will in most cases cause a segfault or similar, but circumstances might also cause different outcomes.

If a malicious user can provide a custom .netrc file to an application or otherwise affect its contents, this flaw could be used as denial-of-service.

CVE-2022-42915: HTTP proxy double-free

If curl is told to use an HTTP proxy for a transfer with a non-HTTP(S) URL, it sets up the connection to the remote server by issuing a CONNECT request to the proxy, and then tunnels the rest of protocol through.

An HTTP proxy might refuse this request (HTTP proxies often only allow outgoing connections to specific port numbers, like 443 for HTTPS) and instead return a non-200 response code to the client.

Due to flaws in the error/cleanup handling, this could trigger a double-free in curl if one of the following schemes were used in the URL for the transfer: dict, gopher, gophers, ldap, ldaps, rtmp, rtmps, telnet

CVE-2022-42916: HSTS bypass via IDN

curl’s HSTS check could be bypassed to trick it to keep using HTTP.

Using its HSTS support, curl can be instructed to use HTTPS directly instead of using an insecure clear-text HTTP step even when HTTP is provided in the URL. This mechanism could be bypassed if the host name in the given URL uses IDN characters that get replaced to ASCII counterparts as part of the IDN conversion. Like using the character UTF-8 U+3002 (IDEOGRAPHIC FULL STOP) instead of the common ASCII full stop (U+002E) ..

Like this: http://curl?se?

Changes

This time around we add one and we remove one.

NPN support removed

curl no longer supports using NPN for negotiating HTTP/2. The standard way for doing this has been ALPN for a long time and the browsers removed their support for NPN several years ago.

WebSocket API

There is an experimental WebSocket API included in this release. It comes in the form of three new functions and a new setopt option to control behavior. The possibly best introduction to this new API is in everything curl.

I am very interested in feedback on the API.

Bugfixes

Here some of the fixed issues from this cycle that I think are especially worthy to highlight.

aws_sigv4 header computation

The sigv4 code got a significant overhaul and should now do much better than before. This is a fairly complicated setup and there are more improvements coming for future releases.

curl man page details multi-use for each option

Every command line option is documented in its own file, which is then used as input when the huge curl.1 man page is generated. Starting now, each such file needs to specify how the option functions when specified more than once. So from now on, this information is mentioned in the man page for all supported options.

deprecate builds with small curl_off_t

Starting in this release, we deprecate support for building curl for systems without 64 bit data types. Those systems are extremely rare this days and we believe it makes sense to finally simplify a few internals when it hurts virtually no one. This is still only deprecated so users can still build on such systems for a short while longer if they really want to.

the ngtcp2 configure option defaults to ‘no’

You need to explicitly ask for ngtcp2 to be enabled in the build.

reject cookie names or content with TAB characters

Cookies with tabs in names or content are not interoperable and they caused issues when curl saved them to disk, so we decided to rather reject them.

for builds with gcc + want warnings, set gnu89 standard

Just to make better sure we maintain compatibility.

use -O2 as default optimize for clang in configure

It was just a mistake that it did not already do this.

warn for –ssl use, considered insecure

To better highlight for users that this option merely suggests for curl that it should use TLS for the protocol, while --ssl-reqd is the one that requires TLS.

ctype functions converted to macros-only

We replaced the entire function family with macros.

100+ documentation spellfixes

After a massive effort and new CI jobs, we now regularly run a spellcheck on most man pages and as a result we fixed lots of typos and we should now be able to better maintain properly spelled documentation going forward.

make nghttp2 less picky about field whitespace in HTTP/2

If built with a new enough nghttp2 library, curl will now ask it to be less picky about trailing white space after header fields. The protocol spec says they should cause failure, but they are simply too prevalent in live servers responses for this to be a sensible behavior by curl.

use the URL-decoded user name for .netrc parsing

This regression made curl not URL decode the user name provided in a URL properly when it later used a .netrc file to find the corresponding password.

make certinfo available for QUIC

The CURLOPT_CERTINFO option now works for QUIC/HTTP/3 transfers as well.

make forced IPv4 transfers only use A queries

When asking curl to use IPv4-only for transfers, curl now only resolves IPv4 names. Out in the wide world there is a significant share of systems causing problems when asking for AAAA addresses so having this option to avoid them is needed.

schannel: when importing PFX, disable key persistence

Some operations when using the Schannel backend caused leftover files on disk afterward. It really makes you wonder who ever thought designing such a thing was a good idea, but now curl no longer triggers this effect.

add and use Curl_timestrcmp

curl now uses this new constant-time function when comparing secrets in the library in an attempt to make it even less likely for an outsider to be able to use timing as a feedback as to how closely a guessed user name or password match the secret ones.

curl: prevent over-queuing in parallel mode

The command line tool would too eagerly create and queue up pending transfers in parallel mode, making a command line with millions of transfers easily use ridiculous amounts of memory.

url parser: extract scheme better when not guessing

A URL has a scheme and we can use that fact to detect it better and more reliable when not using the scheme-guessing mode.

fix parsing URL without slash with CURLU_URLENCODE

When the URL encode option is set when parsing a URL, the parser would not always properly handle URLs with queries that also lacked a slash in the path. Like https://example.com?moo.

url parser: leaner with fewer allocs

The URL parser is now a few percent faster and makes significantly fewer memory allocations and it uses less memory in total.

url parser: reject more bad characters from the host name field

Another step on the journey of making the parser stricter.

wolfSSL: fix session management bug

The session-id cache handling could trigger a crash due to a missing reference counter.

Future

We have several pull-requests in the pipe that will add changes to trigger a minor number bump.

Removals

We are planning to remove the following features in a future-:

• support for systems without 64 bit data type
• support for the NSS TLS library

If you depend on one of those features, yell at us on the mailing list!

Welcome to a new curl release, the result of a slightly extend release cycle this time.

Release presentation

Numbers

the 210th release
3 changes
65 days (total: 8,930)
165 bug-fixes (total: 8,145)
230 commits (total: 29,017)
0 new public libcurl function (total: 88)
2 new curl_easy_setopt() option (total: 299)
0 new curl command line option (total: 248)
79 contributors, 38 new (total: 2,690)
44 authors, 22 new (total: 1,065)
1 security fixes (total: 126)
Bug Bounties total: 40,900 USD

Security

We have yet another CVE to disclose.

control code in cookie denial of service

CVE-2022-35252 allows a server to send cookies to curl that contain ASCII control codes. When such cookies subsequently are sent back to a server, they will cause 400 responses from servers that downright refuse such requests. Severity: low. Reward: 480 USD.

Changes

This release counts three changes. They are:

schannel backend supports TLS 1.3

For everyone who uses this backend (which include everyone who uses the curl that Microsoft bundles with Windows) this is great news: now you too can finally use TLS 1.3 with curl. Assuming that you use a new enough version of Windows 10/11 that has the feature present. Let’s hope Microsoft updates the bundled version soon.

CURLOPT_PROTOCOLS_STR and CURLOPT_REDIR_PROTOCOLS_STR

These are two new options meant to replace and be used instead of the options with the same names without the “_STR” extension.

While working on support for new future protocols for libcurl to deal with, we realized that the old options were filled up and there was no way we could safely extend them with additional entries. These new functions instead work on text input and have no limit in number of protocols they can be made to support.

QUIC support via wolfSSL

The ngtcp2 backend can now also be built to use wolfSSL for the TLS parts.

Bugfixes

This was yet again a cycle packed with bugfixes. Here are some of my favorites:

asyn-thread: fix socket leak on OOM

Doing proper and complete memory cleanup even when we exist due to out of memory is sometimes difficult. I found and fixed this very old bug.

cmdline-opts/gen.pl: improve performance

The script that generates the curl.1 man page from all its sub components was improved and now typically executes several times faster then before. curl developers all over rejoice.

configure: if asked to use TLS, fail if no TLS lib was detected

Previously, the configure would instead just silently switch off TLS support which was not always easy to spot and would lead to users going further before they eventually realize this.

configure: introduce CURL_SIZEOF

The configure macro that checks for size of variable types was rewritten. It was the only piece left in the source tree that had the mention of GPL left. The license did not affect the product source code or the built outputs, but it caused questions and therefore some friction we could easily avoid by me completely writing away the need for the license mention.

close the happy eyeballs loser connection when using QUIC

A silly memory-leak when doing HTTP/3 connections on dual-stack machines.

treat a blank domain in Set-Cookie: as non-existing

Another one of those rarely used and tiny little details about following what the spec says.

configure: check whether atomics can link

This, and several other smaller fixes together improved the atomics support in curl quite a lot since the previous version. We conditionally use this C11 feature if present to make the library initialization function thread-safe without requiring a separate library for it.

digest: fix memory leak, fix not quoted ‘opaque’

There were several fixes and cleanups done in the digest department this time around.

remove examples/curlx.c

Another “victim” of the new license awareness in the project. This example was the only file present in the repository using this special license, and since it was also a bit convoluted example we decided it did not really have to be included.

resolve *.localhost to 127.0.0.1/::1

curl is now slightly more compliant with RFC 6761, follows in the browsers’ footsteps and resolves all host names in the “.localhost” domain to the fixed localhost addresses.

enable obs-folded multiline headers for hyper

curl built with hyper now also supports “folded” HTTP/1 headers.

libssh2:+libssh make atime/mtime date overflow return error

Coverity had an update in August and immediately pointed out these two long-standing bugs – in two separate SSH backends – related to time stamps and 32 bits.

curl_multi_remove_handle closes CONNECT_ONLY transfer

When an applications sets the CONNECT_ONLY option for a transfer within a multi stack, that connection was not properly closed until the whole multi handle was closed even if the associated easy handle was terminated. This lead to connections being kept around unnecessarily long (and wasting resources).

use pipe instead of socketpair on apple platforms

Apparently those platform likes to close socketpairs when the application is pushed into the background, while pipes survive the same happening… This is a change that might be preferred for other platforms as well going forward.

use larger dns hash table for multi interface

The hash table used for the DNS cache is now made larger for the multi interface than when created to be used by the easy interface, as it simply is more likely to be used by many host names then and then it performs better like this.

reject URLs with host names longer than 65535 bytes

URLs actually have no actual maximum size in any spec and neither does the host name within one, but the maximum length of a DNS name is 253 characters. Assuming you can resolve the name without DNS, another length limit is the SNI field in TLS that is an unsigned 16 bit number: 65535 bytes. This implies that clients cannot connect to any SNI-using TLS protocol with a longer name. Instead of checking for that limit in many places, it is now done early.

reduce size of several struct fields

As part of the repeated iterative work of making sure the structs are kept as small as possible, we have again reduced the size of numerous struct fields and rearranged the order somewhat to save memory.

Next

The next release is planned to ship on October 25, 2022.

In the curl project we do timed releases and we try to do them planned and scheduled long in advance. The dates are planned. The content is not.

I have been the release manager for every single curl release done. 209 releases at current count. Having this role means I make sure things are in decent shape for releases and I do the actual mechanic act of running the release scripts etc on the release days.

Since 2014, we make releases on Wednesdays, every eight weeks. We sometimes adjust the date slightly because of personal events (meaning: if I have a vacation when the release is about to happen, we can move it), and we have done several patch releases within a shorter time when the previous release proved to have a serious enough problem to warrant an out-of-schedule release. Even more specifically, I make the releases available at or around 8 am (central euro time) on the release days.

The main objective is to stick to the 56 day interval.

It probably goes without saying but let me be clear: curl is a software project that quite evidently will never be done or complete. It will keep getting fixes, improvements and features for as long as it lives, and as long as it lives we keep making new releases.

Timed releases

We decided to go with a fixed eight weeks release frequency quite arbitrarily (based on past release history and what felt “right”) but it has over time proved to work well. It is short enough for everyone to never have to wait very long for the next release, and yet it is long enough to give us time for both merging new features and having a period of stabilization.

Timed releases means that we ship releases on the predetermined release date with all the existing features and bugfixes that have landed in the master branch in time. If a change is not done in time and it is not merged before the release, it will simply not be included in the release but will get a new opportunity to get included and shipped in the next release. Forever and ever.

I am a proponent of timed releases compared to feature based ones for projects like curl. For the simplicity of managing the releases, for long term planning, for user communication and more.

The cycle has three windows

Within the eight week release cycle, there are three distinct and different windows or phases.

1 – release margin

The cycle can be said to start at the day of a release. The release is created straight from the master git branch. It gets signed, uploaded, blogged about and the news of it is shouted across the globe.

This day also starts the “release margin window”. In this window we still accept and merge bugfixes into the master branch, but we do not merge changes or new features.

The five day margin this speaks of, is that this window gives us a few calendar days to assess and get a feel for how the previous release is being received. Is there a serious bug reported? Did we somehow royally mess up? If we did an important enough snafu, we may decide to do a follow-up patch release soon and if we do, we are in a better position if we have not merged any changes yet into the release branch.

If we decide on doing a patch release, we skip the feature window and go directly to feature freeze.

2 – feature window

If we survived the margin window without anything alarming happening, we open the feature window. On the Monday following the preceding release.

This is the phase during which we merge changes and new features in addition to the regular normal bugfixes (assuming there are any of course). Things that warrant the minor version number for the next release to get bumped.

The feature window is open 23 days. Ideally, people have already been working and polishing on their pull-requests for a while before this window opens and then the work can get merged fairly quickly – and presumably painless.

What exactly can be considered a change and what is a bugfix can of course become a matter of opinion and discussion, but we tend to take the safer approach when in doubt.

3 – feature freeze

In the 28 days before the pending release we only merge bugfixes into the master branch. All features and changes are queued up and will have to wait until the feature window opens again. (Exceptions might be made for experimental and off-by-default features.)

This phase is of course intended for things to calm down, to smooth out rough edges, to fix any leftover mistakes done in the previous feature merges. With the mindset that the pending next release will be the best, most stable, shiniest, glorious release we ever did. With even fewer bugs than before. The next release is always our best release yet.

The release cycle as an image

(Yes, if you squint and roll your chair away from the screen a bit, it looks like the Chrome logo!)

Release frequency graph

The graph below shows the number of days between all curl releases ever done. You can see the eight week release cycle introduced in 2014 visible in the graph, and you can also easily spot that we have done much quicker releases than eight weeks a number of a times since then – all of them actually a sign of some level of failure since that means we felt urged to do a patch release sooner than we had previously anticipated.

Welcome to take the next step with us in this never-ending stroll.

Release presentation

Numbers

the 209th release
8 changes
47 days (total: 8,865)
123 bug-fixes (total: 7,980)
214 commits (total: 28,787)
0 new public libcurl function (total: 88)
2 new curl_easy_setopt() option (total: 297)
1 new curl command line option (total: 248)
51 contributors, 20 new (total: 2,652)
35 authors, 13 new (total: 1,043)
4 security fixes (total: 125)
Bug Bounties total: 34,660 USD

Security

This is another release in which scrutinizing eyes have been poking around and found questionable code paths that could be lead to insecurities. We announce four new security advisories this time – all found and reported by Harry Sintonen. This bumps mr Sintonen’s curl CVE counter up to 17; the number of security problems in curl found and reported by him alone.

CVE-2022-32205: Set-Cookie denial of service

A malicious server can serve excessive amounts of Set-Cookie: headers in a HTTP response to curl and curl stores all of them. A sufficiently large amount of (big) cookies make subsequent HTTP requests to this, or other servers to which the cookies match, create requests that become larger than the threshold that curl uses internally to avoid sending crazy large requests (1048576 bytes) and instead returns an error.

CVE-2022-32206: HTTP compression denial of service

curl supports “chained” HTTP compression algorithms, meaning that a server response can be compressed multiple times and potentially with different algorithms. The number of acceptable “links” in this “decompression chain” was unbounded, allowing a malicious server to insert a virtually unlimited number of compression steps.

CVE-2022-32207: Unpreserved file permissions

When curl saves cookies, alt-svc and hsts data to local files, it makes the operation atomic by finalizing the operation with a rename from a temporary name to the final target file name.

In that rename operation, it might accidentally widen the permissions for the target file, leaving the updated file accessible to more users than intended.

CVE-2022-32208: FTP-KRB bad message verification

When curl does FTP transfers secured by krb5, it handles message verification failures wrongly. This flaw makes it possible for a Man-In-The-Middle attack to go unnoticed and even allows it to inject data to the client.

Changes

We have no less than eight different changes logged this time. Two are command line changes and the rest are library side.

--rate

This new command line option rate limits the number of transfers per time period.

deprecate --random-file and --egd-file

These are two options that have not been used by anyone for an extended period of time, and starting now they have no functionality left. Using them has no effect.

curl_global_init() is threadsafe

Finally, and this should be conditioned to say that the function is only thread-safe on most platforms.

curl_version_info: adds CURL_VERSION_THREADSAFE

The point here is that you can check if global init is thread-safe in your particular libcurl build.

CURLINFO_CAPATH/CAINFO: get default CA paths

As the default values for these values are typically figured out and set at build time, applications might appreciate being able to figure out what they are set to by default.

CURLOPT_SSH_HOSTKEYFUNCTION

For libssh2 enabled builds, you can now set a callback for hostkey verification.

deprecate RANDOM_FILE and EGDSOCKET

The libcurl version of the change mentioned above for the command line. The CURLOPT_RANDOM_FILE and CURLOPT_EGDSOCKET options no longer do anything. They most probably have not been used by any application for a long time.

unix sockets to socks proxy

You can now tell (lib)curl to connect to a SOCKS proxy using unix domain sockets instead of traditional TCP.

Bugfixes

We merged way over a hundred bugfixes in this release. Below are descriptions of some of the fixes I think are particularly interesting to highlight and know about.

improved cmake support for libpsl and libidn2

more powers to the cmake build

address cookie secure domain overlay

Addressed issues when identically named cookies marked secure are loaded over HTTPS and then again over HTTP and vice versa. Cookies are complicated.

make repository REUSE compliant

Being REUSE compliant makes we now have even better order and control of the copyright and licenses used in the project.

headers API no longer EXPERIMENTAL

The header API is now officially a full member of the family.

reject overly many HTTP/2 push-promise headers

curl would accept an unlimited number of headers in a HTTP/2 push promise request, which would eventually lead to out of memory – starting now it will instead reject and cancel such ridiculous streams earlier.

restore HTTP header folding behavior

curl broke the previous HTTP header behavior in the 7.83.1 release, and it has now been restored again. As a bonus, the headers API supports folded headers as well. Folding headers being the ones that are the rare (and deprecated) continuation headers that start with a whitespace.

skip fake-close when libssh does the right thing

Previously, libssh would, a little over-ambitiously, close our socket for us but that has been fixed and curl is adjusted accordingly.

check %USERPROFILE% for .netrc on Windows

A few other tools apparently look for and use .netrc if found in the %USERPROFILE% directory, so by making curl also check there, we get better cross tool .netrc behavior.

support quoted strings in .netrc

curl now supports quoted strings in .netrc files so that you can provide spaces and more in an easier way.

many changes in ngtcp2

There were lots of big and small changes in the HTTP/3 backend powered by ngtcp2.

provide a fixed fake host name in NTLM

curl no longer tries to provide the actual local host name when doing NTLM authentication to reduce information leakage. Instead, curl now uses the same fixed fake host name that Firefox uses when speaking NTLM: WORKSTATION.

return error from “lethal” poll/select errors

A persistent error in select() or poll() could previously be ignored by libcurl and not result in an error code returned to the user, making it loop more than necessary.

strcase optimizations

The case insensitive string comparisons were optimized.

maintain path-as-is after redirects

After a redirect or if doing multi-stage authentication, the --path-as-is status would be dropped.

support CURLU_URLENCODE for curl_url_get

This is useful when for example you ask the API to accept spaces in URLs and you want to later extract a valid URL with such an embedded space URL encoded

Coming next

7.85.0 is scheduled to ship on August 31, 2022.

Welcome to this patch release of curl, shipped only 14 days since the previous version. We decided to cut the release cycle short because of the several security vulnerabilities that were pointed out. See below for details. There are no new features added in this release.

It burns. Mostly in our egos.

Release video

Numbers

the 208th release
0 changes
14 days (total: 8,818)
41 bug-fixes (total: 7,857)
65 commits (total: 28,573)
0 new public libcurl function (total: 88)
0 new curl_easy_setopt() option (total: 295)
0 new curl command line option (total: 247)
20 contributors, 6 new (total: 2,632)
13 authors, 3 new (total: 1,030)
6 security fixes (total: 121)
Bug Bounties total: 22,660 USD

Security

Axel Chong reported three issues, Harry Sintonen two and Florian Kohnhäuser one. An avalanche of security reports. Let’s have a look.

curl removes wrong file on error

CVE-2022-27778 reported a way how the brand new command line options remove-on-error and no-clobber when used together could end up having curl removing the wrong file. The file that curl was told not to clobber actually.

cookie for trailing dot TLD

CVE-2022-27779 is the first of two issues this time that identified a problem with how curl handles trailing dots since the 7.82.0 version. This flaw lets a site set a cookie for a TLD with a trailing dot that then might have curl send it back for all sites under that TLD.

percent-encoded path separator in URL host

In CVE-2022-27780 the reporter figured out how to abuse curl URL parser and its recent addition to decode percent-encoded host names.

CERTINFO never-ending busy-loop

CVE-2022-27781 details how a malicious server can trick curl built with NSS to get stuck in a busy-loop when returning a carefully crafted certificate.

TLS and SSH connection too eager reuse

CVE-2022-27782 was reported and identifies a set of TLS and SSH config parameters that curl did not consider when reusing a connection, which could end up in an application getting a reused connection for a transfer that it really did not expected to.

HSTS bypass via trailing dot

CVE-2022-30115 is very similar to the cookie TLD one, CVE-2022-27779. A user can make curl first store HSTS info for a host name without a trailing dot, and then in subsequent requests bypass the HSTS treatment by adding the trailing dot to the host name in the URL.

Bug-fixes

The security fixes above took a lot of my efforts this cycle, but there were a few additional ones I could mention.

urlapi: address (harmless) UndefinedBehavior sanitizer warning

In our regular attempts to remove warnings and errors, we fixed this warning that was on the border of a false positive. We want to be able to run with sanitizers warning-free so that every real warning we get can be treated accordingly.

gskit: fixed bogus setsockopt calls

A set of setsockopt() calls in the gskit.c backend was fond to be defective and haven’t worked since their introduction several years ago.

define HAVE_SSL_CTX_SET_EC_CURVES for libressl

Users of the libressl backend can now set curves correctly as well. OpenSSL and BoringSSL users already could.

x509asn1: make do_pubkey handle EC public keys

The libcurl private asn1 parser (used for some TLS backends) did not have support for these before.

There has been eight weeks since 7.80.0.

Release presentation

Numbers

the 205th release
1 change
56 days (total: 8,636)
121 bug-fixes (total: 7,518)
189 commits (total: 28,055)
0 new public libcurl function (total: 86)
1 new curl_easy_setopt() option (total: 295)
1 new curl command line option (total: 244)
53 contributors, 25 new (total: 2,558)
32 authors, 14 new (total: 990)
0 security fixes (total: 111)
0 USD paid in Bug Bounties (total: 16,900 USD)

Security

Today we celebrate our fourth consecutive release without any new vulnerability to fix and reveal.

Change

This release comes with just one change to note, but one that brings both a new libcurl setopt (CURLOPT_MIME_OPTIONS) and a new command line option (--form-escape). Starting now, libcurl defaults to percent encoding certain fields when doing multi-part HTTP formposts.

Bug-fixes

As usual, here’s a set of selected favorite bug-fixes of mine from this cycle:

require “see also” for every documented option in curl.1

When the curl command man page is generated at build time, the script now makes sure that there is a “see also” for each option. This will help users find related info. More mandatory information for each option makes us do better documentation that ultimately helps users.

lazy-alloc the table in Curl_hash_add()

The internal hash functions moved the allocation of the actual hash table from the init() function to when the first add() is called to add something to the table. This delay simplified code (when the init function became infallible ) and does even avoid a few allocs in many cases.

enable haproxy support for hyper backend

Plus a range of code and test cases adjusted to make curl built with hyper run better. There are now less than 30 test cases still disabled for hyper. We are closing in!

mbedTLS: add support for CURLOPT_CAINFO_BLOB

Users of this backend can now also use this feature that allows applications to provide a CA cert store in-memory instead of using an external file.

multi: handle errors returned from socket/timer callbacks

It was found out that the two multi interface callbacks didn’t at all treat errors being returned the way they were documented to do. They are now, and the documentation was also expanded to clarify.

nss:set_cipher don’t clobber the cipher list

Applications that uses libcurl built to use NSS found out that if they would select cipher, they would also effectively prevent connections from being reused due to this bug.

openldap: implement STARTTLS

curl can now switch LDAP transfers into LDAPS using the STARTTLS command much like how it already works for the email protocols. This ability is so far limited to LDAP powered by OpenLDAP.

openssl: define HAVE_OPENSSL_VERSION for OpenSSL 1.1.0+

This little mistake made libcurl use the wrong method to extract and show the OpenSSL version at run-time, which most notably would make libcurl say the wrong version for OpenSSL 3.0.1, which would rather show up as the non-existing version 3.0.0a.

sha256/md5: return errors when init fails

A few internal functions would simply ignore errors from these hashing functions instead of properly passing them back to the caller, making them to rather generate the wrong hash instead of properly and correctly returning an error etc.

curl: updated search for a file in the homedir

The curl tool now searches for personal config files in a slightly improved manner, to among other things make it find the same .known_hosts file on Windows as the Microsoft provided ssh client does.

url: check ssl_config when re-use proxy connection

A bug in the logic for checking connections in the connection pool suitable for reuse caused flaws when doing subsequent HTTPS transfers to servers over the same HTTPS proxy.

ngtcp2: verify server certificate

When doing HTTP/3 transfers, libcurl is now doing proper server certificate verification for the QUIC connection – when the ngtcp2 backend is used. The quiche backend is still not doing this, but really should.

urlapi: accept port number zero

Years ago I wrote a blog post about using port zero in URLs to do transfers. Then it turned out port zero did not work like that with curl anymore so work was done and now order is restored again and port number zero is once again fine to use for curl.

urlapi: provide more detailed return codes

There are a whole range of new error codes introduced that help better identify and pinpoint what the problem is when a URL or a part of a URL cannot be parsed or will not be accepted. Instead of the generic “failed to parse URL”, this can now better tell the user what part of the URL that was found out to be bad.

socks5h: use appropriate ATYP for numerical IP address

curl supports using SOCKS5 proxies and asking the proxy to resolve the host name, what we call socks5h. When using this protocol and using a numerical IP address in the URL, curl would use the SOCKS protocol slightly wrong and pass on the wrong “ATYP” parameter which a strict proxy might reject. Fixed now.

Coming up?

The curl factory never stops. There are many pull-requests already filed and in the pipeline of possibly getting merged. There will also, without any doubts, be more ones coming up that none of us have yet thought about or considered. Existing pending topics might include:

• the ManageSieve protocol
• --no-clobber
• CURLMOPT_STREAM_WINDOW_SIZE
• Remove Mesalink support
• HAproxy protocol v2
• WebSockets
• Export/import SSL session-IDs
• HTTP/3 fixes
• more hyper improvements
• CURLFOLLOW_NO_CUSTOMMETHOD

Next release

March 2, 2022 is the scheduled date for what will most probably become curl 7.82.0.

Welcome to another curl release, 7 weeks since the previous one.

Release presentation

Numbers

the 204th release
6 changes
49 days (total: 8,636)
117 bug-fixes (total: 7,397)
198 commits (total: 27,866)
1 new public libcurl function (total: 86)
4 new curl_easy_setopt() option (total: 294)
1 new curl command line option (total: 243)
78 contributors, 44 new (total: 2,533)
50 authors, 28 new (total: 976)
0 security fixes (total: 111)
0 USD paid in Bug Bounties (total: 16,900 USD)

Changes

CURLOPT_MAXLIFETIME_CONN

This new option provides yet another knob for applications to control and limit connection reuse. Using this option, the application sets an upper limit that specifies that connections may not be older than this when being reused. It can for example be used to make sure connections don’t “get stuck” on one single specific load-balancer for extended periods of time.

CURLOPT_PREREQFUNCTION

This new callback gets called immediately before the request is started (= “Pre req”). Gives the application a heads up and some details about what is just about to start.

libssh2: SHA256 fingerprint support

This is a bump up from the previous MD5 version. Make sure that curl connections to the correct host with a cryptographically strong fingerprint check.

curl_url_strerror()

When a URL API function returns an error it does so using a CURLUcode type. Now there’s a function to convert this error code into an error message.

support UNC paths in file: URLs on Windows

The URL parser now understands UNC paths when parsing URLs on Windows.

allow setting of groups/curves with wolfSSL

The wolfSSL backend now allows setting of specific curves for TLS 1.3 connections, which allows users to use post quantum algorithms if wolfSSL is built to support them!

Bug-fixes

This is another release with more than one hundred individual bug-fixes, and here are a selected few I think might be worth highlighting.

more hyper work

I’ve done numerous small improvements in this cycle to take the hyper backend closer to become a solid member of the curl backend family.

print curl --help descriptions aligned right

When listing available options with --help or -h, the list is now showing the descriptions right-aligned which makes the output more easy-to-read in my opinion. Illustration:

store remote IP address for QUIC connections too

HTTP/3 uses QUIC and with this bug fixed, the %{remote_ip} variable for --write-out works there as well – as you’d expect. This fixes the underlying CURLINFO_PRIMARY_IP option.

reject HTTP response codes < 100

The HTTP response parser no longer accepts response code numbers below 100 as a legitimate protocol. The HTTP protocol has never specified any such code so this should not cause any problems. Lots of other HTTP clients already enforce this requirement too.

do not wait for writable socket if there’s no remote HTTP/2 window

If curl runs out of remote HTTP/2 window for a stream while uploading, ie the other end says it can’t receive any data right now, curl would still wait for the socket to be writable which would cause really bad busy-loops.

get libssh2 version at runtime

curl now asks libssh2 for its version string in runtime instead of showing the version that was used back when the curl binary was built, as it might very well be upgraded dynamically after the build!

require all man pages to use the same section headers in the same order

We tighten the bolts even more to make the libcurl documentation consistent. Now all libcurl man pages have to feature the same set of headers in the same order to not cause test failure. This includes a required example section. We also added an extra check to detect a common backslash-wrong-formatting mistake that we’ve previously done several times in man page examples.

NTLM: use DES_set_key_unchecked with OpenSSL

Turns out that the code that is implemented to use OpenSSL for doing NTLM authentication was using a function call that returns error if a “bad” key is used. NTLM v1 being a very weak algorithm in general makes it very easy to end up calling the function with such a weak key and then the NTLM authentication failed…

openssl: if verifypeer is not requested, skip the CA loading

If peer verification is disabled for a transfer when curl is built to use OpenSSL or one of its forks, curl will now completely skip the loading of the CA cert bundle. It was basically only used for being able to show in the verbose output if there was a match or not – that was then ignored anyway – and by skipping the load step the TLS handshake will use less memory and CPU resources.

urlapi: URL decode percent-encoded host names

The URL parser did not accept percent encoded host names. Now it does. Note however that libcurl will not by default percent-encode the host name when extracting a URL for the purpose of keeping IDN names working. It’s a little complicated.

ngtcp2: use QUIC TLS RFC9001

We switch over to use the “real” QUIC identifier when setting up connections instead of using the identifier made for a previous draft version of the protocol. curl still asks h3-29 for HTTP/3 since that RFC has still not shipped, but includes h3 as well – since it turns out some servers assume plain h3 when the final QUIC v1 version is used for transport.

a failed etag save now only fails that transfer

Specifying a file name to save etags in will from now on only fail those transfers using that file. If you specify more transfers that use another file or not use etags at all, those transfers can still get done.

Added test case for checksrc!

The custom tool we use for checking code style compliance, checksrc, has become quite advanced and now checks for a lot of source code details, and when we once again improved it in this release cycle we also added a test case for the tool itself to make sure it maintains its functionality even when we keep improving it going forward!

Next

The next release is planned for January 5, 2022. We have several changes queued up as pull requests already so I’d say it is likely that it then will become version 7.81.0.

Support?

I offer commercial support and curl related contracting for you and your company!

The curl factory has once again cranked out a new curl release.

Release presentation

Numbers

the 202nd release
3 changes
56 days (total: 8,580)
128 bug-fixes (total: 7,270)
186 commits (total: 27,651)
0 new public libcurl function (total: 85)
0 new curl_easy_setopt() option (total: 290)
0 new curl command line option (total: 242)
62 contributors, 25 new (total: 2,484)
41 authors, 16 new (total: 948)
3 security fixes (total: 111)
3,500 USD paid in Bug Bounties (total: 16,900 USD)

Security

This time we once again announce security advisories in association with the release.

CVE-2021-22945 is a double-free flaw in the MQTT code. Patch your old curl or upgrade to this version if you use it to send MQTT. The reporter of this flaw was awarded 1,000 USD from the curl bug-bounty program.

CVE-2021-22946 is a bug in response handling for several protocols (IMAP, POP3 and FTP) that bypasses the enforced TLS check so that even transfers that are explicitly told to require TLS can accidentally silently be performed in clear text! Rewarded 1,000 USD.

CVE-2021-22947 allows a mitm attacker to inject data into the protocol stream for FTP, IMAP, POP3 or SMTP in a way before the TLS upgrade so that curl accepts that data and uses it after after having upgraded to TLS. The untrusted data slips in and gets treated as trusted! Rewarded 1,500 USD.

These two latter ones came as an indirect result/inspiration from the NO STARTTLS research.

Changes

This release comes with three changes to take note of…

Users of the bearssl TLS backend will appreciate that it too now supports the CURLOPT_CAINFO_BLOB option so that the CA certificate easily can be provided in-memory by applications.

The cookie engine in curl now considers http://localhost to be secure and thus cookies that are marked “secure” will be sent over it – even when not using HTTPS. This is done because curl now since a while back makes sure that localhost is always truly local.

Users of the Secure Transport TLS backend can now use CURLINFO_CERTINFO to extract information about the server’s certificate chain.

Bug-fixes

Some of the most interesting bug-fixes we did this round.

use ares_getaddrinfo()

When you build curl to use the c-ares name resolver backend, curl will now use this function to get improved handling for IPv4+ IPv6. This also ups our requirement on c-ares to 1.16.0.

hyper works better

1xx responses, Transfer-Encoding and more have been fixed. The number of tests that are disabled for hyper builds are even fewer than before, but there’s still plenty of work to do before it can be considered not experimental.

cmake builds: avoid poll() on macOS

We have deliberately not used poll() in macOS builds for a long time when building with configure, and now we realized that cmake builds inadvertently had poll() use enabled, which caused curl to misbehave when for example connecting to a host while that connection got closed by the peer. poll() is now disabled on macOS even when cmake is used.

configure: also check lib64 for the OpenSSL pkg-config file

OpenSSL did a very late change just before they shipped version 3.0.0: they modified the default installation path for the library for 64 bit systems from $prefix/lib to $prefix/lib64, and subsequently we had to update our configure script detection logic accordingly. This helps configure to find OpenSSL v3 installs.

curl.1: provide examples for each option

The documentation now must provide at least one example command line for each command line option curl provides. This is verified in the build and will cause build errors if a file doesn’t comply! Feel free to suggest new, more or better examples when you start to see them in the man page.

HTTP 1.1: disallow >3-digit response codes

The HTTP protocol is defined to only allow three-digit numbers and now curl enforces that check stricter. This was in part made to align behavior when curl is built to use hyper.

HTTP 1.1: ignore content-length if any transfer-encoding is used

Non-chunked transfer-encoded content that also sends Content-Length headers is rare but was incorrectly handled by curl. Found when aligning behavior with hyper builds.

http_proxy: only wait for writable socket while sending request

Due to a mistake in the handling of what socket activity to wait for, curl could accidentally be made to busy-loop from the CONNECT request was sent to the proxy until the first data arrived.

Support mbedTLS 3.0.0

When mbedTLS released a new version with support for TLS 1.3 etc, they also modified the API a bit.

Ban strerror

We’ve had our own internal strerror replacement function for a long time (primarily due to it not being thread-safe), but a recent code review revealed that a lot of uses of this function had still crept in. Starting now, our code check tool (checksrc) will error if strerror is used in libcurl code.

The mailing lists move from cool.haxx.se to lists.haxx.se

Our old decommissioned server hosted 29 mailing lists. We moved most of them and killed off a few. All our mailing lists are now hosted on lists.haxx.se, including all the curl related ones of course! The old server name will simply redirect to the new one if you go there with a browser.