Internet. Networking.
I previously mentioned on the libcurl mailing list, that Mark Nottingham in the IETF HTTP Working Group has initiated the work on putting together an overview of all (interesting) existing HTTP implementations
Of course curl is included in the bunch, or rather libcurl, but I would also urge you all to step forward and provide further details on other implementations you worked on or know of!
Next week in Sweden (June 18th), as reported in several places lately including slashdot, the Swedish parliament is supposed to vote for the pretty far-going law allowing FRA (a swedish defence organization previously involved in radio-surveillance etc) to wire-tap phone calls and computer traffic that cross the Swedish borders. The majority in the parliament is for the law, while it seems most of the ordinary people are against it. The hope is now that a few people will vote against their parties, that they will have the guts to stand up and “do the right thing” instead of following the party line.
I won’t go into how silly, stupid and bad such a law is but I’ll instead just show this great video to all swedes:
(video snipped from here)
This banner says (roughly translated by me) “On June 18th the government will take away your personal integrety. All internet traffic, all phone calls, all email and SMS traffic will be wire-tapped starting January 1st 2009. Big brother sees you! … and violates the Swedish Constitution.”
I noticed the new site publicsuffix.org that has been setup by the mozilla organization in an attempt to list public suffixes for all TLDs in the world, to basically know how to prevent sites from setting cookies that would span over just about all sites under that “public suffix”.
While I can see what drives this effort and since we have the same underlying problem in curl as well, I have sympathy for the effort. Still, I dread “having to” import and support this entire list in curl only to be able to better work like the browsers in the cookie department. Also, it feels like a cat and mouse race where the list may never be complete anyway. It is doomed to lack entries, or in the worst case list “public suffixes” that aren’t any such public suffixes anymore and thus it’ll prevent sites using that suffix to properly use cookies…
There’s no word on the site if IE or Opera etc are going to join this effort.
Update: there are several people expressing doubts about the virtues of this idea. Like Patrik Fältström on DNSOP.
In case you haven’t read it before, Randy Bush‘s 55 page PDF slide show named “IPv6 Transition & Operational Reality” is a harsh (but quite accurate) description of how the IPv6 protocol was made, where some of its major problems lie and why the transition is going so slow etc.
I tried to find some official and recent figures or statements from some of the more IPv6-positive people and companies, but I failed to find much updates from after the year 2000 or so…
Speaking of network things that aren’t so successfully deployed: DNSSEC. Apparently iis.se (runs the Swedish TLD) tested 10 broadband routers (article and PDF in Swedish only) how well they support this (I believe mainly because .se tries to be a pioneer in DNSSEC), and 7 of the tested ones failed… Personally I’ve never liked the fact that DNSSEC isn’t really crafted to do it securely all the way.
As I wrote a while ago, companies using http proxies make people feel a need to break out of their proxies.
Bryan is a friend who recently found out that his company is switching proxy to a different one and apparently both corkscrew and proxytunnel have problems with this new piece, and since libcurl offers quite a lot of functionality to accomplish almost this, a new project was born: curltunnel.
One immediate benefit of using libcurl is the support for multiple authentication methods, in fact more than any of the above mentioned tools.
However, it seems our first quick stab at making this tool (currently 278 lines of code), made it work for several common cases but… not for Bryan’s new proxy.
The current theory is that the proxy actually checks for SSL traffic and only lets that through, and thus it prevents the ssh server banner to appear when we try to tunnel through the proxy to a remote ssh server on port 443. If further testing proves this correct, we will of course have to add a SSL layer to the mix.
My wife wants to keep some videos found on youtube, and I really can’t recommend just keeping bookmarks to a random web site like that. Not if you want the content to be available in a few years ahead, or even ten or twenty years. Then downloading the files to keep the locally is the only sane way to make it somewhat more reliable.
To download the files you can do it with a browser or with a command line tool:
Browser Style
Command Line Style
There exist several command line tools “out there” that do the job. I tried youtube-dl and it did the job splendidly by only proving the main HTTP URL on the command line.
The main lacking feature is that it names the output flv based on the ‘v’ variable in the URL so the downloads end up being named things like “f_8wuVEYMZ8.flv”…
Play the local FLV movies
For this, I can only recommend the lovely VLC media player, available on all modern platforms.
During the summer 2001, me and my wife toured Vietnam and we had a great time. For that occasion I set up a little online diary that would allow us to post entries while on the road, to allow our families and friends back home to be able to keep up with what we were doing.
Fast forward to present day: the diary “submit new entry” form is still left on my site, and while it no longer works (it hasn’t worked for many years) – it is still one of the most visited pages on my site! It seems the automated spam bots find it and submit crap to it… the crap doesn’t end up anywhere to be seen nor is it even stored on the server, but it clearly identifies evil machines! Isn’t that a honeypot as good as any?!
So far during September 2007, no less than 309 unique IP addresses have issued a POST on that page..
Far too often we end up behind a proxy that limits our network access in one or more ways. There are however clever ways that in most cases allow us to work around the nuisances the proxies impose, and I’ve written down my “guide” on how to do it here!
This is a procedure I’ve used myself many times and I’ve ended up explaining it to others several times as well so I felt it was about time I wrote it down.
I’ve tried to catch most quirks and be detailed and accurate, but please point out if you find any errors or mistakes in there. I hope to be able to perhaps add more specific config examples and command lines as well to make it even easier to follow.