curl with partial files

Back in September 2023, we extended the curl command line tool with a new fairly advanced and flexible variable system. Using this, users can use files, environment variables and more in a powerful way when building curl command lines in ways not previously possible – with almost all existing command line options.

curl command lines were already quite capable before this, but these new variables certainly took it up several additional notches.

Come February 2025

In the pending curl 8.12.0 release, we extend this variable support a little further. Starting now, you can assign a variable to hold the contents of a partial file. Get a byte range from a given file into a variable and use that variable in the command line, instead of using the entire file.

You can get the first few bytes and use as a username, you can get a hundred bytes in the middle of a file and POST that or do countless other things.

Byte range

You ask curl to read a byte range from a file instead of the whole one by appending [n-M] to the variable name, when you assign a variable. Where N and M are the first and the last byte offsets into the file, 0 being the first byte. If you omit the second number, it means until the end of file.

For example, get the first 32 bytes from a file named secret and set as password for daniel:

curl --variable "pwd[0-31]@secret" \
--expand-user daniel:{{pwd}} \
https://example.com/

Skip the first thousand bytes from a file named localfile and send the rest of it in a POST:

curl --variable "upload[1000-]@localfile" \
--expand-post '{{upload}}' \
https://example.com/

With functions

You can of course also combine the byte offsets with the standard expand functions. For example, get the first hundred bytes from the file called random and send them base64 encoded in a POST:

curl --variable "binary[0-99]@random" \
--expand-post '{{binary:b64}}' \
https://example.com/

I hope you will like it.

Update

After his post was first published, we discussed the exact syntax for this feature and decided to tweak it a little to make it less likely that old curl versions could be tricked when trying a new command line options.

This version is now showing the updated syntax.

2 thoughts on “curl with partial files”

  1. Small error: In the last example you write ‘from the file called random’ but in the curl command you read from localfile.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.