The curl command line option --write-out or just -w for short, is a powerful and flexible way to extract information from transfers done with the tool. It was introduced already back in version 6.5 in the early 2000.

This option takes an argument in which you can add “variables” that hold all sorts of different information, from time information, to speed, sizes, header content and more.

Some users have right out started to use the -w output for logging of the performed transfer, and when you do that there was a little detail missing: the ability to output the time the transfer completed. After all, most log lines actually feature the time in one way or another.

Starting in curl 8.16.0, curl -w knows the time and now allows the user to specify exactly how to output that time in the output. Suddenly this output is a whole notch better for logging purposes.

%time{format}

Since log files also tend to use different time formats I decided I didn’t want to use a fixed format and risk that a huge portion of users will think it is the wrong one, so I went straight with strftime formatting: the user controls the time format using standard %-flags: different ones for year, month, day, hour, minute, second etc.

Some details to note:

1. The time is provided using UTC, not local.
2. It also supports %f for microseconds, which is a POSIX extension already used by Python and possible others
3. %z and %Z (for time zone offset and name) had to be fixed to become portable and identical across systems and platforms

Example

Here’s a sample command line outputting the time the transfer completed:

curl -w "%time{%a %b %e %Y - %H:%M:%S.%f} %{response_code}\n" https://example.com -o saved

When I ran this command line it gave me this output:

Wed Aug 6 2025 - 12:43:45.160382 200

Credits

The clock image by Alexa from Pixabay

In the early days of curl development we (I suppose it was me personally but let’s stick with we so that I can pretend the blame is not all on me) made the possibly slightly unwise decision to make the -X option change the HTTP method for all requests in a curl transfer, even when -L is used – and independently of what HTTP responses the server returns.

That decision made me write blog posts and inform people all over about how using -X superfluously causes problems.

In curl 8.16.0, we introduce a different take on the problem, or better yet, a solution really: a new command line option that offers a modified behavior. Possibly the behavior people were thinking curl was having all along.

Just learn to use --follow going forward (in curl 8.16.0 and later).

This option works fine together with -X and will adjust the method in the possible subsequent requests according to the HTTP response code.

A long time ago I wrote separately about the different HTTP response codes and what they mean in terms of changing (or not) the method.

–location remains the same

Since we cannot break existing users and scripts, we had to leave the exiting --location option working exactly like it always has. This option is this mutually exclusive with --follow, so only pick one.

QUERY friendly

Part of the reason for this new option is to make sure curl can follow redirects correctly for other HTTP methods than the good old fashioned GET, POST and PUT. We already see PATCH used to some extent but perhaps more important is the work on the spec for the new QUERY method. It is a flavor of POST, but with a few minor but important different properties. Possibly enough for me to write a separate blog post about, but right now we can stick to it being “like POST”, in particular from a HTTP client’s perspective.

We want curl to be able to do a “post” but with a QUERY method and still follow redirects correctly. The -L and -X combination does not support this.

curl can be made to issue a proper QUERY request and follow redirects correctly like this:

curl -X QUERY --follow -d sendthis https://example.com/

Thank you for flying curl!

I’m convinced a lot of people have not yet figured out that curl has supported parallel downloads for six years already by now.

Provided a practically unlimited number of URLs, curl can be asked to get them in a parallel fashion. It then makes sure to keep N transfers alive for as long as there is N or more transfers left to complete, where X is a custom number but 50 by default.

Concurrently transferring data from potentially a large number of different hosts can drastically shorten transfer times and who doesn’t prefer to complete their download job sooner rather than later?

Limit connections per host

At times however, you may want to do a lot of transfers, and you want to do them in parallel for speed, but maybe you prefer to limit how many connections curl should use per each hostname among all the URLs?

This per-host limit is a feature libcurl has offered applications for a long time and now the time has come for curl tool users to also enjoy its powers.

Per host should perhaps be called per origin if we spoke web lingo, because it rather limits the number of connections to the same protocol + hostname + port number. We call that host here for simplicity.

To set a cap on how many connections curl is allowed to use for each specific server use --parallel-max-host [number].

For example, if you want to download ten million images from this site, but never use more than six connections:

curl --parallel --parallel-max-host 6 https://example.com/[1-10000000].jpg --remote-name

Connections

Pay special attention to the exact term: this limits the number of connections used to each host. If the transfers are done using HTTP/2 or HTTP/3, they can be done using many streams over just one or a few connections so doing 50 or 200 transfers in parallel should still be perfectly doable even with a limited number of connections. Not so much with HTTP/1.

Ships in 8.16.0

This command line option will become available in the pending curl version 8.16.0 release.

We have always had a custom command line option parser in curl. It is fast and uncomplicated and gives us the perfect mix of flexibility and function. It also saves us from importing or using code with another license.

In one aspect it has behaved slightly different than many other command line parsers: the way it accepts arguments to long options.

Long options are the options provided using a name that starts with two dashes and are often not single-letters. Example:

curl --user-agent "curl/2000" https://example.com/

The example above tells curl to use the user agent curl/2000 in the transfer. The argument provided to the --user-agent option is provided separated with a space.

When instead using the short version of the same option, the argument can be specified with a space in between or not:

curl -A curl/2000 https://example.com/

or

curl -Acurl/2000 https://example.com/

What about equals sign?

A common paradigm and syntax style for accepting long options in command line tools is the “equals sign” approach. When you provide an argument to a long option you do this by appending an equals sign followed by the argument to the option; with no space. Like this:

curl --user-agent="curl/2000" https://example.com/

This example uses double quotes but they are of course not necessary if there is no space or similar in the argument.

Bridging the gap

To make life easier for future users, curl now also support this latter style – starting in curl 8.16.0. With this syntax supported, curl accepts a more commonly used style and therefore should induce less surprises to users. To make it easier to write curl command lines.

I emphasize that change this is an improvement for future users, because I really don’t think it is a good idea for most user to switch to this syntax immediately. This of course because all the older curl versions that are still used widely around the word do not support it.

I think it is better if we wait a year or two until we start using this option style in curl documentation and example command lines. To give time for users to upgrade to a version that has support for it.

Downloading data from a remote URL is probably the single most common operation people do with curl.

Often, users then add various additional options to the command line to extract information from that transfer but may also decide that the actually fetched data is not interesting. Sometimes they don’t get the accurate meta-data if the full download is not made, sometimes they run performance measurements where the actual content is not important, and so on. Users sometimes have reasons for not saving their downloads.

They do downloads where the actual downloaded content is tossed away. On GitHub alone, we can find almost one million command lines doing such curl invokes.

curl of course offers multiple ways to discard the downloaded data, but the maybe most straight-forward way is to write the contents to a null device such as /dev/null on *nix systems or NUL: on windows. Like this:

curl https://example.com/ --output /dev/null

or using the short option

curl https://example.com/ -o /dev/null

In many cases we can accomplish the same thing with a shell redirect – which also redirects multiple URLs at once:

curl https://example.com/ >/dev/null

Improving nothing

The command line above is perfectly fine and works fine and has been doing so for decades. It does however have two drawbacks:

1. Lack of portability. curl runs on most operating systems and most options and operations work identically, to the degree that you can often copy command lines back and forth between machines without thinking much about it. Outputting data to /dev/null is however not terribly portable and trying that operation on Windows for example will cause the command line to fail.
2. Performance. It may not look like much, but completely avoiding writing the data instead of writing it to /dev/null makes benchmarks show a measurable improvement. So if you don’t want the data, why not do the operation faster rather than slower?

The shell redirect approach has the same drawbacks.

Usage

The new option is used as follows, where it needs one --out-null occurrence per URL it wants to redirect.

curl https://example.com/ --out-null

This allows you to for example send one to null and save the other:

curl https://example.com/ --out-null https://example.net/ --output save-data

Coming in 8.16.0

This command line option debuts in curl 8.16.0, shipping in September 2025.

Credits

Stefan Eissing brought this option. He also benchmarked this option.

Back in September 2023, we extended the curl command line tool with a new fairly advanced and flexible variable system. Using this, users can use files, environment variables and more in a powerful way when building curl command lines in ways not previously possible – with almost all existing command line options.

curl command lines were already quite capable before this, but these new variables certainly took it up several additional notches.

Come February 2025

In the pending curl 8.12.0 release, we extend this variable support a little further. Starting now, you can assign a variable to hold the contents of a partial file. Get a byte range from a given file into a variable and use that variable in the command line, instead of using the entire file.

You can get the first few bytes and use as a username, you can get a hundred bytes in the middle of a file and POST that or do countless other things.

Byte range

You ask curl to read a byte range from a file instead of the whole one by appending [n-M] to the variable name, when you assign a variable. Where N and M are the first and the last byte offsets into the file, 0 being the first byte. If you omit the second number, it means until the end of file.

For example, get the first 32 bytes from a file named secret and set as password for daniel:

curl --variable "pwd[0-31]@secret" \
     --expand-user daniel:{{pwd}} \
     https://example.com/

Skip the first thousand bytes from a file named localfile and send the rest of it in a POST:

curl --variable "upload[1000-]@localfile" \
     --expand-post '{{upload}}' \
     https://example.com/

With functions

You can of course also combine the byte offsets with the standard expand functions. For example, get the first hundred bytes from the file called random and send them base64 encoded in a POST:

curl --variable "binary[0-99]@random" \
     --expand-post '{{binary:b64}}' \
     https://example.com/

I hope you will like it.

Update

After his post was first published, we discussed the exact syntax for this feature and decided to tweak it a little to make it less likely that old curl versions could be tricked when trying a new command line options.

This version is now showing the updated syntax.

This is episode four in my mini-series about shiny new features in the upcoming curl 8.10.0 release.

One of the most commonly used curl command line options is the dash capital O (-O) which also is known as dash dash remote-name (--remote-name) in its long form.

This option tells curl to create a local file using the name from the filename part of the provided URL when downloading. I.e. when you tell curl

curl -O https://example.com/file.html

This command line conveniently creates a local file called file.html in which it saves the downloaded data.

The -O option has been supported with this functionality since curl first shipped, in March 1998. An important point here is that it picks the name from the URL so that a user can tell what filename it creates. No surprises. The remote server is not involved in naming it.

What about no filename scenarios?

URLs do not necessarily need to have filename parts. Like these examples:

http://example.com/
http://example.com/path/
http://example.com/one/two/?id=12345

Since there are no filename parts in these URLs, they used to cause curl to refuse to operate with -O and instead return error. curl could not create a local filename to use:

$ curl -O http://example.com/
curl: Remote filename has no length
curl: (23) Failed writing received data to disk/application

Trying harder

Starting in curl 8.10.0, curl works a little harder to come up with a filename to store the download in when -O is used. While there is no filename part in the URL, the user did ask curl to download the URL to a local file so it now tries a few extra steps:

1. Use the filename part from the URL if there is one, like before.
2. If there is no filename but there is a path provided in the URL, extract the right-most directory name from the URL and use as filename.
3. If there is neither a filename nor a path in the URL, curl uses a default, fixed, filename as a final backup: curl_response. This name intentionally has no extension because curl has no idea what data that will come and using an extension could mislead users into believing it says something about the type of content.

Several people have insisted that index.html would be better and sensible default file name. I cannot agree with that, since it might just as well be an image or a tarball of your favorite open source project. I think naming such a file index.html would be more misleading than simply sticking to the neutral curl_response.

Let me give you a little table showing what filenames that will be used with curl -O and a given set of URLs:

URL	local filename
http://example.com/one.html	one.html
http://example.com/one.html?clues=no	one.html (curl ignores the query part)
http://example.com/one/two/?id=42	two (because it is the right-most directory piece)
http://example.com/path/	path (because it is the right-most directory piece)
http://example.com/	curl_response (because no filename nor directory to use)

Find out which name

You can use curl’s -w, –write-out option and its %{filename_effective} variable to learn exactly which name that was used.

Prefer another name?

There is always the -o (lowercase o) option that lets you specify whatever filename you like. You do not have to let curl pick the filename for you.

Clobber or not

curl will by default overwrite, clobber if you will, any previously existing file using the same name. If you rather curl took a more careful approach, consider using –no-clobber in your command lines. It makes curl pick an alternative filename if the chosen one already exists when curl is about to download data into a local file.

This is episode three in my mini-series of posts describing news in the coming curl 8.10.0 release. Part one was more help, part two verbose, verbose and verbosest.

This new command line option in curl 8.10.0 is a simple one that has been requested by users repeatedly over the years so I figure it was about time we actually provide it.

If the target file already exists on disk, skip downloading it.

It is exactly as simple as that. No date check, no size check, no checking if the file is even what you want it to be. If the target file is present and exists that is a signal enough that the file should not be downloaded; to skip the transfer.

A real-world command line using this feature could then look like this:

curl --skip-existing --output local/dir/file https://example.com

Or if instead -O is used, it still works the same:

curl --skip-existing -O https://example.com/me.jpg

Easy, right? See the manpage.

Broken files can also be present

To avoid a previous broken download remainder to linger around and cause future transfers to get skipped, remember that curl also has a –remove-on-errror option.

Ships

In curl 8.10.0, on September 11, 2024.

Image

From a movie with a suitable if even perhaps subtle reference.

A key feature for a tool like curl is its ability to help the user diagnose command lines and operations that do not work the way the user intended them to.

When I do XYZ, why does it not work?

The command line option -v and its longer version --verbose have been supported by curl since day one for this purpose. A boolean flag that when used shows what is going on by outputting extra information from the execution.

I need to emphasize the boolean part here. Up until curl 8.10.0, this option was a plain boolean. You either did not get verbose output or you got it. There was no levels or ways to increase or decrease the amount of information shown. Just a binary one or zero. On or off.

But starting in 8.10.0 the story is different.

The world

Meanwhile, there is a universe of additional command line tools out there. Many other tools also offers a -v option for outputting verbose tracing information. In many other tools, the -v is not a boolean but instead you might get additional output if you add more vs. -vv shows a little more, -vvv even more etc.

Users are fairly trained on this. To the extent that we often get to see users use -vvv etc on curl command lines in bug reports etc. The curl command line parser accepts more of them fine (any amount really), but repeating them just enables the boolean again and again to no extra effect.

When we asked users for their favorite command line option in the annual curl user survey in May 2024, a noticeable amount of respondents said -vv or -vvv. Even though they do nothing extra than -v. I think this shows to which extent people are trained to and are used to having these options for command line tools.

Make curl do what you think it did

Therefore.

In curl 8.10.0, coming on September 11, 2024, we introduce support for -vv, -vvv and -vvvv. One, two, three or four vs. (Maybe we add more in the future.)

If you write the v-letters consecutive next to each other like that, you increase the logging level; the amount of verbose output curl shows. It might then possibly do something in the style that many users already expected it to do.

The extra logging it can do in 8.10.0 is actually nothing new, what is new is that you can get to it by simply using -vv and friends. The old style of getting such extra verbose tracing is to instead use a selected combination of –trace-time, –trace-ascii and –trace-config.

Backwards compatibility

In curl we care deeply about backwards compatibility and not breaking users existing scripts and use cases when we ship new versions. This change is perhaps on the border of possibly doing this, so we have tried to tread as gently as we can to make sure that risk is slim.

This is why doing something like curl -v -v will not increase the level, because a user might have one of the switches in ~/.curlrc, another one on the command line and a third one in a custom file read with curl’s -K option etc. We want the extra output level to be explicitly asked for.

Using a single -v after a -vv or -vvv resets the level back to the original lowest-but-enabled for the same reason. The --no-verbose option also still works, even though the option is not strictly a boolean anymore and curl normally otherwise only supports the --no- prefix for boolean command line options.

Credits

Stefan Eissing wrote the pull-request that landed this change.