{"id":3796,"date":"2012-05-12T22:22:08","date_gmt":"2012-05-12T20:22:08","guid":{"rendered":"http:\/\/daniel.haxx.se\/blog\/?p=3796"},"modified":"2023-12-23T23:32:39","modified_gmt":"2023-12-23T22:32:39","slug":"shorter-http-requests-for-curl","status":"publish","type":"post","link":"https:\/\/daniel.haxx.se\/blog\/2012\/05\/12\/shorter-http-requests-for-curl\/","title":{"rendered":"shorter HTTP requests for curl"},"content":{"rendered":"\n<p>Starting in curl 7.26.0 (due to be released at the end of May 2012), we will shrink the <a href=\"http:\/\/en.wikipedia.org\/wiki\/User_agent\"><em>User-agent<\/em>: header<\/a> that <a href=\"https:\/\/curl.se\/\">curl<\/a> sends by default in HTTP(S) requests to something much shorter! I suspect that this will raise some eyebrows out there so even though I&#8217;ve emailed about it to the <a href=\"https:\/\/curl.se\/mail\/list.cgi?list=curl-users\">curl-users list<\/a> before I thought I&#8217;d better write it up and elaborate.<\/p>\n\n\n\n<p>A default &#8216;curl localhost&#8217; on Debian Linux makes 170 bytes get sent in that single request:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<pre class=\"wp-block-preformatted\">GET \/ HTTP\/1.1\nUser-Agent: curl\/7.24.0 (i486-pc-linux-gnu) libcurl\/7.24.0 OpenSSL\/1.0.0g zlib\/1.2.6 libidn\/1.23 libssh2\/1.2.8 librtmp\/2.3\nHost: localhost\nAccept: *\/*<\/pre>\n<\/blockquote>\n\n\n\n<p>As you can see, the user-agent description takes up a large portion of that request, and this for really no good reason at all. Without sacrificing any functionality I shrunk the same request down to 71 bytes:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<pre class=\"wp-block-preformatted\">GET \/ HTTP\/1.1\nUser-Agent: curl\/7.24.0\nHost: localhost\nAccept: *\/*<\/pre>\n<\/blockquote>\n\n\n\n<p>That means we shrunk it down to 41% of the original size. I&#8217;ll admit the example is a bit extreme and most other normal use cases will use longer host names and longer paths, but even for a URL like &#8220;<a href=\"https:\/\/daniel.haxx.se\/docs\/curl-vs-wget.html\">https:\/\/daniel.haxx.se\/docs\/curl-vs-wget.html<\/a>&#8221; we&#8217;re down to 50% of the original request size (100 vs 199).<\/p>\n\n\n\n<p>Can we shrink it even more? Sure, we could leave out the version number too. I left it in there now only to allow some kind of statistics to get extracted. We can&#8217;t remove the entire header, we need to include a user-agent in requests since there are too many servers who won&#8217;t function properly otherwise.<\/p>\n\n\n\n<p>And before anyone asks: this change is only for the curl command line tool and not for <a href=\"https:\/\/curl.se\/libcurl\/\">libcurl<\/a>, the library. libcurl does in fact not send any user-agent at all by default&#8230;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Starting in curl 7.26.0 (due to be released at the end of May 2012), we will shrink the User-agent: header that curl sends by default in HTTP(S) requests to something much shorter! I suspect that this will raise some eyebrows out there so even though I&#8217;ve emailed about it to the curl-users list before I &hellip; <a href=\"https:\/\/daniel.haxx.se\/blog\/2012\/05\/12\/shorter-http-requests-for-curl\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">shorter HTTP requests for curl<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,13],"tags":[33,230,219],"class_list":["post-3796","post","type-post","status-publish","format-standard","hentry","category-curl","category-net","tag-curl-and-libcurl","tag-http","tag-network"],"_links":{"self":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/3796","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/comments?post=3796"}],"version-history":[{"count":18,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/3796\/revisions"}],"predecessor-version":[{"id":23776,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/posts\/3796\/revisions\/23776"}],"wp:attachment":[{"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/media?parent=3796"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/categories?post=3796"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/daniel.haxx.se\/blog\/wp-json\/wp\/v2\/tags?post=3796"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}