Tag Archives: Open Source

cURL and libcurl, Open Source

Code re-use is fun

Back in 2003 I wrote up support for the HTTP NTLM authentication method for libcurl. Happy with my achievement, I later that year donated a GPL licensed version of my code to the Wget project (which also was my first contact with the signed paper stuff with the GNU/FSF to waive my copyright claims and instead hand them over). What was perhaps not so amusing with this code was when both curl and Wget 2005 were discovered to have the same security flaw due to my mistakes in this code shared by both projects!

Just recently, the neon project seems to be interested in taking on the version I adjusted somewhat for them, so possibly the third HTTP code is soon using this. Yeah I posted it on their mailing list back then so it has been sitting there in the archives maturing for some 6 years by now…

I also happened to fall over the SSH Tunnel Creator tool, which I’ve never used myself, that apparently snatched my neon donation (quite according to what the license allowed of course) and used it in their tool to do NTLM!

It’s actually not until recent years I discovered libntlm, and while I don’t know how good it was back in the days when I wrote my first NTLM stuff I generally think using existing libs is the better idea…

Open Source, Technology

A stream of streamings

I’m a last.fm fan. I love its ability to not only stream music without needing a dedicated client installed (yes a flash application I think suits a purpose) and I think it’s ability to provide music I might also like is amazingly nice. I’m a “random it all” kind of guy when I listen to my local music collection in most situations as well. It is not specifly well suited for listening on exact the songs you want, as if you select a specific song it won’t even play the full-length version of it.

Lately there’s been a lot of buzz in Swedish tech media about spotify, which is a similar idea (at the moment still an on invitation-only thing in Sweden). They stream music, but only to a proprietary Windows or Mac client and currently they offer free listening with ads (embedded in the audio and visible in the client) or 99 SEK (== 9 Euros == 11 USD) per month. The client is highly focused on specific songs or artists and it has nothing in the way of “random artitists I generally like and similar ones”. I’m not too thrilled.

Spotify offers its service in several places, and I hear in the UK it’s not even invitation-only (which of course is useful for the more forward-thinking hacking kind of guys who thus use a UK based proxy to reach them). There’s however no sign of a Linux client. We’re forced to run their windows client with Wine.

I’ve gotten the impression that Pandora is a similar concept to play with if you happen to be based in the US. I’m in Sweden and Pandora just shows me a “We are deeply, deeply sorry to say that due to licensing constraints, we can no longer allow access to Pandora for listeners located outside of the U.S.”

The other day despotify.se showed up. A bunch of clever hackers reverse engineered the Spotify protocol and stream and offer a full unofficial open sourced ncurses/libvorbis/pulse-audio/gstreamer/expat/zlib/openssl-based player! Reading the code shows that these guys certainly had to crack some hard nuts, but the activity in their IRC channel seems fierce and the code is rather clean so I expect it to turn out to eventually become a fine player if Spotify just doesn’t decide to play hard ball with them. Unfortunately, despotify hasn’t yet been able to produce a single sound for me since it has just died on assert()s on basically any attempts I’ve tried. The interface is also a bit… strange and not the easiest to figure out. (It should be noted that the despotify client still requires you who have an actual spotify account.)

It’ll be interesting to see how Spotify, or perhaps the big media companies owning all the music rights, will act on this initiative. This client does open up abilities for new fancy features. How about ripping the stream? How about re-distributing the stream like as a proxy? And of course it being open, it does open up for adding features I want to add.

Update: just hours after I posted this, Spotify closed access to their service using the despotify client as long as you’re not a “premium” (paying) user…

Open Source

Project is Standard now

I fell over a (warning: Swedish!) article on the Swedish idg.se site describing the nominees for the yearly award “Guldmusen” (“the golden mouse”). One of them is this year the highly deserving Adam Dunkel, originator of among other things the very cool lwip (light weight IP) open source project.

However, in both articles on IDG that’s not how lwip is described. It is instead claimed to be “an unofficial world standard”.

“world standard” huh? Yes I admit that makes it sound quite a few notches better than “well-used and appreciated open source project”.

But lwip is an TCP/IP stack. A very small one. How can that be a standard, even if you call it an unofficial one?

idg.se continues to be rediculed by me and my friends and the reasons are silly things like this. Let’s hope they continue to amuse us for a long time to come! 🙂

Open Source

Open source personal

I participate in a range of different open source projects. Of course I spend more time on some of them and only a very little time in most of them, but I’m currently listed as member of 18 projects on sourceforge and 16 on ohloh and I can easily figure out a bunch more than aren’t listed on either of those sites.

I’m just the kind of guy who tend to actually get the code and write up a patch for problems, and in fact also in many cases I’ll write an fresh application and publish it openly for the world (not that my typical programs get any particularly large audience but still). I’m not saying everyone has to be like this, I’m just describing me here.

It seems this is a troublesome concept for people to grasp.

I get a large amount of private mail where people talk about “your project” (as in a single one that I am supposed to understand which one they’re referring to) and just about all open source-related interview/questionnaire things I’ve filled in tend to assume My One Single Project. In the first case I can often guess which one they refer to by the phrasing of the mail, and in the second case I tend to answer for the project I’m involved the most in.

So I get this feed of private emails on projects I participate in, but I don’t like private emails about open source projects when people request and expect free support and help. If they want free support, I expect the people to post the questions publicly and open to allow others to reply and read both the question and the subsequent answer online, right there at the time they’re asked but also much later when searching for help on the same subject as then the answers will be around in mailing list archives etc.

These days I have a blanket reply form that I bounce back when I get private support mails and I will admit that most people respect that after having been told about the situation. Every now and then of course I get a violent refusal for sympathy and instead I get to learn I’m an arrogant bastard. This is also related to the fact that:

We (Haxx) run and offer commercial support around curl and libcurl, and for that purpose we have a dedicated support email address. Mail there if you’re willing to pay for support. That’s actually quite clearly spelled out everywhere where that address is displayed, but yet people seem to find that a good place to mail random questions and bug reports. Just today I got a very upset mail response after I mentioned the “paid support” part of the deal there expecting us (me?) to instantly fix bugs regardless since I’ve been told about them per email…

All in all, I’m not really complaining since I’m generally getting along fine with everyone and stuff around this.

Just everyone try to keep things apart: the projects, the people and the companies. They’re sometimes intertwined but sometimes not.

Development, Open Source

Open source in my day job

From people in the open source community and then especially friends and fellow hackers in projects I am involved in, I sometimes get questions on how my open source participations affect my “real job”.

I work as a consultant during the days and I do contract development for hire. I’ve been a consultant since 1996 and during this time I’ve participated in more than 25 “full-time” projects for almost as many customers.

I contribute to numerous open source projects and I’ve done it for many years. I lead and maintain several open source projects. I’ve committed many thousands of times to public source code repositories.

Does the contributions make me more attractive to potential customers? Not particularly is my rather sad experience. While some of my customers notice my track record (my CV does of course mention my most notable contributions) most of my day-job clients focus solely on other paid projects I’ve done and that exact technologies and products I worked with and created in the past. It may of course not be too strange as things I do and get paid for is then potentially “good enough for someone to pay me for” while the stuff I do for free in open source projects are… well, not paid for and thus it can’t be qualified by that ruler.

Do I get new assigments thanks to my open source projects? Yes, I do, but usually they tend to be on the smallish side and not of the bigger kinds my regular assignments at work are.

And the reality is of course also that the vast vast vaaast majority of all software consultants that people hire to do development have no public record of open source involvement so it could just be a result of that this is so rare the customers never had a reason to learn or adapt to using open source contributions as a “factor”.

Or maybe I’m just ignorant and haven’t figured out how my customers truly work.

Do I work with open source in my day job? Yes almost exclusively. I’ve been working with Linux in various embedded systems basically the last 8 years and working with Linux systems pretty much implies a wide range of open source development tools as well.

Development, Open Source

4 ohloh improvements I’d like

I am a stats junkie so I like my stats in large amounts. But I like the stats to be right and as accurate as possible, and when I look at what ohloh produces I like the concepts and ideas in general, I just think their implementation is lacking in a few vital areas that need improvement:

1. There are no dependencies or hierarchies between packages, so “I use this” counters get worthless since people mark end-user packages they use. Low-level support packages and libraries that are used indirectly don’t get many “use counts”

2. Doing very few commits in a very well used project with few authors gives you way way more points than doing a bus-load of commits in something less used with many fellow contributors. This makes the top-list of people very skewed as some of the top-64 people only did a few hundred commits ever. I doubt many mortals would consider someone who only ever did 300 commits to be a top community person. At the very moment I write this, the #1 ranked person has done 20 commits during 5 months…!

3. Too few versioning systems are supported, leaving out huge chunks of the open source world. Bazaar, mercurial and a few more are a bit too popular to be ignored without the results getting skewed.

4. I’d like to see the “number of users” of products as a percentage, as the total number of users they show include all contributors to all projects. Out of the 140,000 users (which undoubtedly include a lot of duplicates), it would surprise me if more than 10,000 have actually registered what products they use. I’ve tried to find the exact number but I failed. So 3,000 users don’t mean 3,000 out of 140,000 but 3,000 out of 10,000…

Web

My Firefox Add-ons

3 Comments

I simply need to have this list somewhere so that I can find out my own add-ons again when I’m running Firefox away from home!

Adblock Plus – since ads are too annoying these days

DownThemAll – because I like to be able to get whole batches of images or similar at times

Fission – just a silly eye-candy thing

Forecastfox – I like weather forecasts!

FoxClocks – helps me keep track of the time my friends around the world have at different moments.

It’s All Text – makes web based editing/posting a more pleasurable experience by allowing me to edit such contents with emacs!

Live HTTP Headers is a must when you want to figure out how to repeat your browser’s actions with a set of curl commands.

Open in Browser allows me to open more stuff within the browser itself, even when the Content-Type is bad.

Right-Click-Link is great when you quickly want to browse to links you find in plain text sections.

Torbutton lets me quickly switch to anonymous browsing.

User Agent Switcher lets me trick stupid server-side scripts into beleiving I use a different browser or even operating system.

What great add-ons did I miss?

(Some nitpickers would say that I don’t run Firefox since I use Debian and then it is called Iceweasel, but while that is entirely true, Iceweasel is still the Firefox source code and the Add-ons are in fact still Firefox Add-ons even if they also run perfectly fine on Iceweasel.)

Development, Open Source

My million users

I’ve been working professionally with computers since 1991 and explicitly as a developer since 1993. I’ve written one or two lines of code since then. How many users could there be out there that are using something that includes my code?

Open source

I’ve participated in a wide range of open source projects, so of course all direct users of those projects would count: curl, Rockbox and let’s include subversion and others. I would guess that there are at least one million users of curl, quite likely more than so of subversion and Rockbox may also reach a million users or so. It’s of course impossible to know for sure…

Lots of open source projects use libraries that I work on now and have worked with in the past. Primarily libcurl and c-ares. Such as Boinc, git, bazaar, darcs. Millions of users, no doubt (Boinc alone has some 1.5 million users). The OLPC’s XO laptop comes with (lib)curl. I think most Linux distros these days come with curl installed. How many linux installations are there? libcurl is rather popular when used within PHP as well and there are many many million installations of PHP out there. I have code in wget, also used by millions.

Closed source users of open source I’ve participated in

Adobe acrobat reader (for non-windows platforms), Adobe’s flash player and various other Adobe products, Second life, Google Earth and others. They’re bound to have several million users. curl is included in Mac OS X.

There are also a lot of devices that use libcurl that are even harder to track: Sandisk makes mp3 players that use libcurl, Sony makes a video device that uses libcurl, Tilgin, Neuros and others make IPTV-devices that use libcurl. libcurl is used for multiple “installers” such as the one AOL provide for a specific router. There are many company users.

Closed source stuff I’ve worked with on my day-job

… is of course also used widely and all over, but me being an embedded guys I mostly work on software in products and most of the products I’ve worked within have been for various niche markets in which I have little or no knowledge about how much the products (and thus my code) are actually used. I’ve left my fingerprints on several networking products, IPTV/Digital TV settop boxes, railroad equipments, a car ignition tester, 3g/telecom switches, rfid receivers, laser-using positioning systems and more.

How many millions?

Ok, let’s for the sake of the argument say that there’s somewhere around 100 million devices with my code from me included – I really have no idea how to make a sensible estimate. Let’s for simplicity also say that there are 100 million users of these devices. I would also guess that about half of the world’s population isn’t near using devices I may have programmed. Thus, if you’re using “devices” in general there’s a probability of 3 billion/100 million = 1/30 that you’re using something that includes code that I’ve worked on…

In fact, that number is then valid for any random “device” user – if you’re reading this on my blog I don’t expect you to be very random but rather a specialized person and then I would say the likeliness of you having at least something with my code in it is almost 100% guaranteed…

Where would you say my biggest weaknesses in this reasoning are?

Blogging, Open Source

FNOSS hosts nordic foss blogs

5 Comments

There’s yet another blog aggregator on the internet now, and this time it’s fnoss.org which includes blogs from a bunch of “Nordic” (I would assume that means people from the northern parts of Europe) people writing about free software and related matters. I am one.

My blog is since previously also seen in the advogato aggregation.

This of course makes my blog get more read but like the rss feeds it also makes it harder for me to know how many readers/visitors I have since it’s all distributed. Not that this number matter very much anyway…