{"id":26207,"date":"2024-12-30T10:46:15","date_gmt":"2024-12-30T09:46:15","guid":{"rendered":"https:\/\/daniel.haxx.se\/blog\/?p=26207"},"modified":"2024-12-30T15:58:01","modified_gmt":"2024-12-30T14:58:01","slug":"curl-with-partial-files","status":"publish","type":"post","link":"https:\/\/daniel.haxx.se\/blog\/2024\/12\/30\/curl-with-partial-files\/","title":{"rendered":"curl with partial files"},"content":{"rendered":"\n

Back in September 2023, we extended the curl command line tool with a new fairly advanced and flexible variable system<\/a>. Using this, users can use files, environment variables and more in a powerful way when building curl command lines in ways not previously possible – with almost all existing command line options.<\/p>\n\n\n\n

curl command lines were already quite capable before this, but these new variables certainly took it up several additional notches.<\/p>\n\n\n\n

Come February 2025<\/h2>\n\n\n\n

In the pending curl 8.12.0 release, we extend this variable support a little further. Starting now, you can assign a variable to hold the contents of a partial<\/em> file. Get a byte range from a given file into a variable and use that variable in the command line, instead of using the entire file.<\/p>\n\n\n\n

You can get the first few bytes and use as a username, you can get a hundred bytes in the middle of a file and POST that or do countless other things.<\/p>\n\n\n\n

Byte range<\/h2>\n\n\n\n

You ask curl to read a byte range from a file instead of the whole one by appending [n-M]<\/strong><\/code> to the variable name, when you assign a variable. Where N and M are the first and the last byte offsets into the file, 0 being the first byte. If you omit the second number, it means until the end of file.<\/p>\n\n\n\n

For example, get the first 32 bytes from a file named secret<\/em> and set as password for daniel<\/em>:<\/p>\n\n\n\n

curl --variable \"pwd[0-31]@secret\" \\
     --expand-user daniel:{{pwd}} \\
     https:\/\/example.com\/<\/pre>\n\n\n\n
Skip the first thousand bytes from a file named localfile<\/em> and send the rest of it in a POST:<\/p>\n\n\n\n
curl --variable \"upload[1000-]@localfile\" \\
     --expand-post '{{upload}}' \\
     https:\/\/example.com\/<\/pre>\n\n\n\n
With functions<\/h2>\n\n\n\n
You can of course also combine the byte offsets with the standard expand functions<\/em>. For example, get the first hundred bytes from the file called random<\/em> and send them base64 encoded in a POST:<\/p>\n\n\n\n
curl --variable \"binary[0-99]@random\" \\
     --expand-post '{{binary:b64}}' \\
     https:\/\/example.com\/<\/pre>\n\n\n\n
I hope you will like it.<\/p>\n\n\n\n
Update<\/h2>\n\n\n\n
After his post was first published, we discussed the exact syntax for this feature and decided to tweak it a little to make it less likely that old curl versions could be tricked when trying a new command line options.<\/p>\n\n\n\n
This version is now showing the updated syntax.<\/p>\n","protected":false},"excerpt":{"rendered":"
Back in September 2023, we extended the curl command line tool with a new fairly advanced and flexible variable system. Using this, users can use files, environment variables and more in a powerful way when building curl command lines in ways not previously possible – with almost all existing command line options. curl command lines … Continue reading curl with partial files<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":5,"featured_media":16486,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[214,33],"class_list":["post-26207","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-curl","tag-command-line","tag-curl-and-libcurl"],"_links":{"self":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/26207","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/comments?post=26207"}],"version-history":[{"count":23,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/26207\/revisions"}],"predecessor-version":[{"id":26240,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/26207\/revisions\/26240"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/media\/16486"}],"wp:attachment":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/media?parent=26207"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/categories?post=26207"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/tags?post=26207"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}