Category Archives: Web

web stuff

Network, Web

More suggested HTTP fun

I’ve already previously expressed my deepest dislike with where the HTML5 work is going, and just yesterday two new internet-drafts appeared on ietf.org that spurred up discussions all around. They’re claimed to be “part of our effort to remove from HTML5 sections that are more appropriate elsewhere” but I’m thinking they’re rather inappropriate everywhere…

The first one named Content-Type Processing Model hits a subject that I’ve been over before, namely the stupidity of having web browsers guess the content based on what it looks like. IE introduced the “I really mean it property“, the HTML5 team wants to standardize the way of the guessing. Personally, I think the world of web will become a better place if the browsers would instead become stricter and more closer follow what the servers actually say the contents is, and then all users would complain to the site admins if things are wrong and then things should be fixed.

Guessing content types allows for sloppy behaviors, it makes it harder to write browsers for the web and it still features a significant risk of guessing wrong.

The second draft propagates for the new HTTP header “Origin”, which according to the authors would help to guard servers against CSRF (“Cross-Site Request Forgery“). The main author says 3% of users on the Internet gets their Referer header stripped while virtually none gets Origin stripped. I claim this is a bogus argument since they strip Referer beacause it is a known and established header and Origin is not. I also completely fail to see the goodness of this and based on several of the other responses on the ieth-http-wg mailing list I am not alone…

cURL and libcurl, Development, Linux, Network, Web

IETF http-state group created

Over at the IETF another group was just created named http-state (with an associated mailing list) with the specific goal:

Ultimately, the purpose of this group is to create an updated HTTP State Management Mechanism RFC (aka cookies) that will supersede the Netscape spec, RFCs 2109, 2964, 2965 then add in real-world usage (e.g. HTTPOnly), and possibly add in additional features and possibly merge in draft-broyer-http-cookie-auth-00.txt and draft-pettersen-cookie-v2-03.txt.

I’ve joined the list and I hope to follow and participate in this, as I believe the current state of HTTP cookies is a rather sorry mess and the Netscape spec is still what closest describes how cookies work in the wild. Of course I’ll do it with my libcurl experience in my luggage.

While it perhaps would be cool to join the group in more formal way, there’s no way for me to participate in that IETF meeting in San Francisco in March.

Network, Security, Web

Fun with executable extensions in viewvc

A few years ago I wrote up silly little perl script (let’s call it script.pl) that would fetch a page from a site that returns a “random URL off the internet”. I needed a range of URLs for a test program of mine and just making up a thousand or so URLs is tricky. Thus I wrote this script that I would run and allow to get a range of URLs on each invoke and then run it again later and append to the log file. It wasn’t a fancy script, but it solved my task.

The script was part of a project I got funded to work on, that was improving libcurl back in 2005/2006 so I thought adding and committing the script to CVS felt only natural and served a good purpose. To allow others to repeat what I did.

Fast forward to late 2008. The script is now browsable via viewvc on a site that… eh, doesn’t have “.pl” disabled as a cgi extension in its config! The result of course is that each time someone tries to view the script using the web interface, the web server invokes the script locally!

All of a sudden I get a mail from someone, who apparently is admin or something of the site this old script was using, and he mentions that a machine on our network is hammering his site with many requests per second (38 requests/second apparently) and asked me to stop this. It turns out a search engine crawler has indexed the viewvc output several times, and now some 8 processes or so were running this script.pl and they were all looping around getting a page, outputting the URL, getting another page…

While I think 38 requests second is a bit low to even be considered a DOS, it certainly wasn’t intended nor friendly and I was greatly surprised when I slowly realized how it all came to end up like this! Man I suck! It reminds me of my other extension mess from just a few months ago…

Maybe I’ll learn how to do things right in the future when I grow up!

Blogging, Web

Avatars by gravatar

Daniel's gravatar avatar imageI’m using one of those fancy WordPress plugins on this blog that makes use of gravatar for the avatar images that appear next to your name when you post a comment. So if you comment here on daniel.haxx.se and want to see a fancy personal image next to your wise words, skip over to gravatar.com and put up a picture of you that then will be associated with your email address.

This system does not reveal your email address to any outsider, as the avatar is received from their service simply by sending a oneway hash of your address.

This isn’t really anything new here, it’s been like this for a while but I figured I should explain it better to the few who might not have realized this yet…

Network, Web

Filling our pipes

1 Comment

At around 13:43 GMT Friday the 5th of December 2008, the network that hosts a lot of services like this site, the curl site, the rockbox site, the c-ares site, CVS repositories, mailing lists, my own email and a set of other open source related stuff, become target of a vicious and intense DDoS attack. The attack was in progress until about 17:00 GMT on Sunday the 7th. The target network is owned and ran by CAG Contactor.

Tens of thousands of machines on the internet suddenly started trying to access a single host within the network. The IP they targeted has in fact never been publicly used as long as we’ve owned it (which is just a bit under two years) and it has never had any public services.

We have no clue whatsoever why someone would do this against us. We don’t have any particular services that anyone would gain anything by killing. We’re just very puzzled.

Our “ISP”, the guys we buy bandwidth and related services from, said they used up about 1 gigabit/sec worth of bandwidth and with our “mere” 10 megabit/sec connection it was of course impossible to offer any services while this was going on.

It turns out our ISP did the biggest blunder and is the main cause for the length of this outage: we could immediately spot that the target was a single IP in our class C network. We asked them to block all traffic to this IP as far out as possible to stop such packets from entering their network. And they did. For a short while there was silence and sense again.

For some reason that block “fell off” and our network got swamped again and it then remained unusable for another 48 hours or so. We know this, since our sysadmin guy investigated our firewall logs on midday Sunday and they all revealed that same target IP as destination. Since we only have a during-office-hours support deal with our network guys (as we’re just a consultant company with no services that really need 24 hour support) they simply didn’t care much about our problem but said they would deal with it Monday morning. So our sysadmin shutdown our firewall to save our own network from logging overload and what not.

Given the explanations I’ve got over phone (I have yet to see and analyze logs from this), it does sound like some sort of SYN flood and they attempted to connect to many different TCP ports.

4-5 hours after the firewall was shutdown, the machines outside of our firewall (but still on our network) suddenly became accessible again. The attack had stopped. We have not seen any traces of it since then. The firewall is still shutdown though, as the first guy coming to the office Monday morning will switch it on again and then – hopefully – all services should be back to normal.

Web

My Firefox Add-ons

3 Comments

I simply need to have this list somewhere so that I can find out my own add-ons again when I’m running Firefox away from home!

Adblock Plus – since ads are too annoying these days

DownThemAll – because I like to be able to get whole batches of images or similar at times

Fission – just a silly eye-candy thing

Forecastfox – I like weather forecasts!

FoxClocks – helps me keep track of the time my friends around the world have at different moments.

It’s All Text – makes web based editing/posting a more pleasurable experience by allowing me to edit such contents with emacs!

Live HTTP Headers is a must when you want to figure out how to repeat your browser’s actions with a set of curl commands.

Open in Browser allows me to open more stuff within the browser itself, even when the Content-Type is bad.

Right-Click-Link is great when you quickly want to browse to links you find in plain text sections.

Torbutton lets me quickly switch to anonymous browsing.

User Agent Switcher lets me trick stupid server-side scripts into beleiving I use a different browser or even operating system.

What great add-ons did I miss?

(Some nitpickers would say that I don’t run Firefox since I use Debian and then it is called Iceweasel, but while that is entirely true, Iceweasel is still the Firefox source code and the Add-ons are in fact still Firefox Add-ons even if they also run perfectly fine on Iceweasel.)

IT Politics, Security, Web

Snooping on government HTTPS

As was reported by some Swedish bloggers, and I found out thanks to kryptoblog, it seems the members of the Swedish parliament all access the internet via a HTTP proxy. And not only that, they seem to access HTTPS sites using the same proxy and while a lot of the netizens of the world do this, the members of the Swedish parliament have an IT department that is more big-brotherish than most: they decided they “needed” to snoop on the network traffic even for HTTPS connections – and how do you accomplish this you may ask?

Simple! The proxy simply terminates the SSL connection, then fetches the remote HTTPS document and run-time generates a “faked” SSL cert for the peer that is signed by a CA that the client trusts and then delivers that to the client. This does require that the client has got a CA cert installed locally that makes it trust certificates signed by the “faked” CA but I figure the parliament’s IT department “help” its users to this service.

Not only does this let every IT admin there be able to snoop on user names and passwords etc, it also allows for Man-In-The-Middle attacks big-time as I assume the users will be allowed to go to HTTPS sites using self-signed certificates – but they probably won’t even know it!

The motivation for this weird and intrusive idea seems to be that they want to scan the traffic for viruses and other malware.

If I were a member of the Swedish parliament I would be really upset and I would uninstall the custom CA and I would seriously consider accessing the internet using an ssh tunnel or similar. But somehow I doubt that many of them care, and the rest of them won’t be capable to take counter-measures against this.

cURL and libcurl, Linux, Web

Solaris 10 ships libcurl

I fell over this document named “What’s New in the Solaris 10 10/08 Release” and it includes this funny little quote towards the end:

C-URL – The C-URL Wrappers Library

C-URL is a utility library that provides programmatic access to the most common Internet protocols such as, HTTP, FTP, TFTP, SFTP, and TELNET. C-URL is also extensively used in various applications.

The project is cURL, the tool is curl and the library is libcurl. There’s nothing named C-URL and it isn’t any “wrappers library”… And the list of protocols is also funny since it includes 6 protocols while a modern libcurl supports 13 different ones, and also if you build libcurl to support SFTP you also get SCP (which the list doesn’t include) etc.

It just looks so very sloppy to me. But hey, what do I know?

Mail, Web

Please hide my email

1 Comment

… I don’t want my employer/wife/friends to see that I’ve contributed something cool to an open source project, or perhaps that I said something stupid 10 years ago.

I host and co-host a bunch of different mailing list archives for projects on web sites, and I never cease to get stumped by how many people are trying hard to avoid getting seen on the internet. I can understand the cases where users accidentally leak information they intended to be kept private (although the removal from an archive is then not a fix since it has already been leaked to the world), but I can never understand the large crowd that tries to hide previous contributions to open source projects because they think the current or future employers may notice and have a (bad) opinion about it.

I don’t have the slightest sympathy for the claim that they get a lot of spam because of their email on my archives, since I only host very public lists and the person’s address was already posted publicly to hundreds of receivers and in most cases also to several other mailing list archives.

People are weird!