{"id":12086,"date":"2019-03-12T22:49:36","date_gmt":"2019-03-12T21:49:36","guid":{"rendered":"https:\/\/daniel.haxx.se\/blog\/?p=12086"},"modified":"2019-03-13T09:54:35","modified_gmt":"2019-03-13T08:54:35","slug":"looking-for-the-refresh-header","status":"publish","type":"post","link":"https:\/\/daniel.haxx.se\/blog\/2019\/03\/12\/looking-for-the-refresh-header\/","title":{"rendered":"Looking for the Refresh header"},"content":{"rendered":"\n

The other day someone filed a bug on curl that we don’t support redirects with the Refresh header<\/a>. This took me down a rabbit hole of Refresh header research and I’ve returned to share with you what I learned down there.<\/p>\n\n\n\n

tl;dr Refresh is not a standard HTTP header.<\/p>\n\n\n\n

As you know, an HTTP redirect is specified to use a 3xx response code and a Location: header<\/a> to point out the new URL (I use the term URL here but you know what I mean). This has been the case since RFC 1945 (HTTP\/1.0). According to an old mail from Roy T Fielding<\/a> (dated June 1996), Refresh “didn’t make it” into that spec. That was the first “real” HTTP specification. (And the HTTP we used before 1.0<\/a> didn’t even have headers!)<\/p>\n\n\n\n

The little detail that it never made it into the 1.0 spec or any later one, doesn’t seem to have affected the browsers. Still today, browsers keep supporting the Refresh header<\/a> as a sort of Location: replacement even though it seems to never have been present in a HTTP spec.<\/p>\n\n\n\n

In good company<\/h2>\n\n\n\n

curl is not the only HTTP library that doesn’t support this non-standard header. The popular python library requests apparently doesn’t according to this bug from 2017<\/a>, and another bug was filed<\/a> about it already back in 2011 but it was just closed as “old” in 2014.<\/p>\n\n\n\n

I’ve found no support in wget or wget2 either for this header.<\/p>\n\n\n\n

I didn’t do any further extensive search for other toolkits’ support, but it seems that the browsers are fairly alone in supporting this header.<\/p>\n\n\n\n

How common is the the Refresh header?<\/h2>\n\n\n\n

I decided to make an attempt to figure out, and for this venture I used the Rapid7 data trove<\/a>. The method that data is collected with may not be the best – it scans the IPv4 address range and sends a HTTP request to each TCP port 80, setting the IP address in the Host: header. The result of that scan is 52+ million HTTP responses from different and current HTTP origins. (Exactly 52254873 responses in my 59GB data dump, dated end of February 2019).<\/p>\n\n\n\n

Results from my scans<\/h4>\n\n\n\n