Tag Archives: c-ares

c-ares, cURL and libcurl, Network

C-ares, now and ahead!

3 Comments

The project c-ares started many years ago (March 2004) when I decided to fork the existing ares project to get the changes done that I deemed necessary – and the original project owner didn’t want them.

I did my original work on c-ares back then primarily to get a good asynchronous name resolver for libcurl so that we would get around the limitation of having to do the name resolves totally synchronously as the libc interfaces mandate. Of course, c-ares was and is more than just name resolving and not too surprisingly, there have popped up other projects that are now using c-ares.

I’m maintaining a bunch of open source projects, and c-ares was never one that I felt a lot of love for, it was mostly a project that I needed to get done and when things worked the way I wanted them I found myself having ended up as maintainer for yet another project. I’ve repeatedly mentioned on the c-ares mailing list that I don’t really have time to maintain it and that I’d rather step down and let someone else “take over”.

After having said this for over 4 years, I’ve come to accept that even though c-ares has many users out there, and even seems to be appreciated by companies and open source projects, there just isn’t any particular big desire to help out in our project. I find it very hard to just “give up” a functional project, so I linger and do my best to give it the efforts and love it needs. I very much need and want help to maintain and develop c-ares. I’m not doing a very good job with it right now.

Threaded name resolves competes

I once thought we would be able to make c-ares capable of becoming a true drop-in replacement for the native system name resolver functions, but over the years with c-ares I’ve learned that the dusty corners of name resolving in unix and Linux have so many features and fancy stuff that c-ares is still a long way from that. It has also made me turn around somewhat and I’ve reconsidered that perhaps using a threaded native resolver is the better way for libcurl to do asynchronous name resolves. That way we don’t need any half-baked implementations of the resolver. Of course it comes at the price of a new thread for each name resolve, which turns really nasty of you grow the number of connections just a tad bit, but still most libcurl-using applications today hardly use more than just a few (say less than a hundred) simultaneous transfers.

Future!

I don’t think the future has any radical changes or drastically new stuff in the pipe for c-ares. I think we should keep polishing off bugs and add the small functions and features that we’re missing. I believe we’re not yet parsing all records we could do, to a convenient format.

As usual, a project is not about how much we can add but about how much we can avoid adding and how much we can remain true to our core objectives. I wish the growing popularity will make more people join the project and then not only to through a single patch at us, but to also hand around a while and help us somewhat more.

Hopefully we will one day be able to use c-ares instead of a typical libc-based name resolver and yet resolve the same names.

Join us and help us give c-ares a better future!

c-ares

Open Source, Web

roffit lives!

Many moons ago I created a little tool I named roffit. It is just a tiny perl script that converts a man page written in the nroff format to good-looking HTML. I should perhaps also add that I didn’t find any decent alternatives then so I wrote up my own version. I’ve been using it since in projects such as curl, c-ares and libssh2 to produce web versions of the docs.

It has just done its job and I haven’t had any needs to fiddle with it. The project page lists it as last modified in 2004, even though I actually moved it to a sourceforge CVS repo back in 2007.

Just a few days ago, I got emailed and was notified that Debian has it included as a package in the distribution and someone was annoyed on some particular flaws.

This resulted in a bunch of bugs getting submitted to the Debian bug tracker, I started up the brand new roffit-devel mailing list to easier host roffit discussions and I switched over the CVS repo to a git one on github.

If you like seeing man pages turned into web pages, consider joining up and help us improve this thing!

Network, Security

The big protocols

1 Comment

OWASP Sweden once again arranged another interesting meeting, this time with three talks.owasp

The title of the meeting on January 21st here in Stockholm called the protocols “the big ones” (but in Swedish) but I have no idea what kind of measurement they’ve used or what the small ones are or what other “big protocols” there might be! 😉

First we got to hear HÃ¥vard Eidnes tell us about BGP and that protocol seems to suffer from its share of security problems with the protocol itself but perhaps even more with the actual implementations as one of the bigger recent BGP-related incidents that was spoken about was about how internal routes were leaked to the outside from Pakistan in Feb 2008 which made them block the entire world’s access to Youtube. This talk also gave us some insights on the “wild west” of international routing and the lack of control and proper knowledge about who’s allowed to route what to where.

There then was a session by Rickard Bellgrim about DNSSEC and even though I’ve heard talks about this protocol in the past I couldn’t but to again feel that man they have a lot of terminology in that world that makes even a basic description fairly hard to keep up with in some parts of it all. And man do they have a lot of signing and keys and fingerprints and trusts going on… Of course DNSSEC is the answer to lots of existing problems with DNS and DNSSEC certainly opens up a range of new fun. The idea to somehow replace the need for ca-certs by storing keys in DNS is interesting, but even though technically working and sound I fear the browser vendors and the CAs of the SSL world won’t be very fast to turn the wheels to roll in that direction. DNSSEC certainly makes name resolving a lot more complicated, and I wonder if c-ares should ever get into that game… And BTW, DNSSEC of course doesn’t take away the fact that specific implementations may still be vulnerable to security flaws.

The last talk of the evening was about SSL, or rather TLS, held by Fredrik Hesse. He gave us a pretty detailed insight into how the protocol works, and then a fairly detailed overview of the flaws discovered during the last year or so, primarily MD5 and rogue ca certs, the null-prefix cert names and the TLS renegotiation bug. I felt good about already knowing just about everything of what he told us. I can also boast with having corrected the speaker afterward at the pub where we were having our post-talk-beers as he was evidently very OpenSSL focused when he spoke about what SSL libraries can and cannot do.

A great evening. And with good beers too. Thanks to the organizers!

c-ares, Open Source

c-ares 1.7.0

The first c-ares release so far in 2009 took place today when we shipped c-ares 1.7.0 and uploaded it to the web site.

News this time include:

  • Added ares_library_init() and ares_library_cleanup()
  • Added ares_parse_srv_reply(), ares_parse_txt_reply() and ares_free_data()
  • in6_addr is not used in ares.h anymore, but a private ares_in6_addr is
    instead declared and used
  • ares_gethostbyname() now supports ‘AF_UNSPEC’ as a family for resolving
    either AF_INET6 or AF_INET
  • a build-time configured ares_socklen_t is now used instead of socklen_t
  • new –enable-curldebug configure option
  • ARES_ECANCELLED is now sent as reason for ares_cancel()
  • new –enable-symbol-hiding configure option
  • new Makefile.msvc for any MSVC compiler or MS Visual Studio version
  • addrttl and addr6ttl structs renamed to ares_addrttl and ares_addr6ttl
  • naming convention for libraries built with MSVC, see README.msvc

The set of bugfixes done include these:

  • ares_parse_*_reply() functions now return ARES_EBADRESP instead of
    ARES_EBADNAME if the name in the response failed to decode
  • only expose/export symbols starting with ‘ares_’
  • fix \Device\TCP handle leaks triggered by buggy iphlpapi.dll
  • init without internet gone no longer fails
  • out of bounds memory overwrite triggered with malformed /etc/hosts file
  • function prototypes in man pages out of sync with ares.h

As usual, c-ares would be nothing without the fierce and skillful help provided by a team of volunteer hackers. We always need more help and assitance, join the c-ares mailing list and join in the fun!

c-ares

c-ares, cURL and libcurl, libssh2, Open Source

Adapting to being behind

1 Comment

For many years I’ve always kept up to speed with my commitments in my primary open source projects. I’ve managed to set aside enough time to close the bug reports as fast as they have poured in. This, while still having time to work on new features every now and then.

During this last year (or so) however, I’ve come to realize that I no longer can claim to be in that fortunate position and I now find myself seeing the pile of open bugs get bigger and bigger over time. I get more bug reports than I manage to close.

There are of course explanations for this. In both ends of the mix actually. I’ve got slightly less time due my recent decision to go working for Haxx full-time, and how I’ve decided to focus slightly more on paid work which thus leads to me having less time for the unpaid work I’m doing.

Also, I’ve seen activity raise in the curl project, in the libssh2 project and in the c-ares project. All of these projects have the same problem of various degrees: a lack of participating developers working on fixing bugs. Especially bugs reported by someone else.

Since this situation is still fairly new to me, I need to learn on how to adapt to it. How to deal with a stream of issues that is overwhelming and I must select what particular things I care about and what to “let through”. This of course isn’t ideal for the projects but I can’t do much more than proceed to the best of my ability, to try to make people aware of that this is happening and try to get more people involved to help out!

Don’t get fooled by my focus on “time” above. Sometimes I even plainly lack the energy necessary to pull through. It depends a lot on the tone or impression I get from the report or reporter how I feel, but when a reporter is rude or just too “demanding” (like constantly violating the mailing list etiquette or just leaving out details even when asked) I can’t but help to feel that at times working as a developer during my full-day paid hours can make it a bit hard to then work a couple of hours more in the late evening debugging further.

The upside, let’s try to see it as a positive thing, is that now I can actually “punish” those that clearly don’t deserve to get helped since I now focus on the nice people, the good reports, the ones which seem to be written by clever people with an actual interest to see their problems addressed. Those who don’t do their part I’ll just happily ignore until they shape up.

I will deliberately just let issues “slip through” and not get my attention and require that if they are important enough people will either report it again, someone else will step up and help fix them or perhaps someone will even consider paying for the fix.

cURL and libcurl, Development, Open Source

How much for a bug?

7 Comments

no bugsWarning: blog post with no clear conclusion!

I offer support deals to companies that want to get help with Open Source programs I’ve contributed to. The deals I’ve made so far have primarily involved libcurl, c-ares or libssh2, but that’s basically because those are projects in which I participate a lot in (and maintain) so people find me easily in relation to those projects.

I wouldn’t mind accepting service and support deals for other projects or software products either, as long as they are products I know and am fairly familiar with already and I am not scared of digging in and fixing things under the hood when that is required.

In fact, I could very well consider to offer to fix bugs in any Open Source software. Like a general: if you have a bug in an open source project that you really want fixed and you can’t do it yourself I might be your man. Of course this would be limited to some certain kinds of projects and programs, but it could still include a wide range of software. A lot more than the ones I happen to be involved in at any particular point in time.

But while “a bug” is a fairly easily defined term to a user who can’t make something work in a given program it can be anything from dead simple to downright impossible for a developer to fix. The fact that users many times cannot determine if a “bug” is hard or easy, if it’s a bug or a feature not working on purpose, makes such a business deal very hard to provide.

How to pay to get a bug fixed?

Fixed price per bug? Presumably only tricky bugs would be considered for this so it would require a fairly high fixed price. But then it’ll also never be used for simple bugs either since the fixed price would scare away such use cases. I don’t think a fixed-price scheme works very well for this.One dollar bill

Then we only have a variable price approach left. A common way for a consultant like me is to charge for my time spent on a project: I set an hourly rate, I fix the issue in N hours. I charge hourly rate * N. For smallish projects, this is less attractive to customers. If we have no previous relationship, there’s a trust issue where the customer might not just blindly accept that I worked 10 hours on a task they think sounds easy so they feel overcharged. Also, there’s the risk that I estimate the job to be 2 hours but end up spending 12. My conclusion is that per-hour pricing doesn’t work for this either.

A variable price approach based on something else than number of hours it took for me to fix the problem is therefore needed.

A bug fix is of course worth whatever someone is willing to pay for it. But we don’t know what they are prepared to pay. On the other end, a bug fix can get done by someone for the price he/she is willing to accept to get the job done. So where is the cross section of those two unknown graphs?

I don’t have the answer here. I’m very interested in feedback and suggestions though. If you would pay for a bug fix, how would you like to get the price set?

c-ares, cURL and libcurl, libssh2, Open Source, Rockbox, Web

50 hours offline

1 Comment

Several sites in the haxx.se domain and other stuff related to me and my fellows were completely offline for almost 50 hours between August 24th 19:00 UTC and August 26th 20:30 UTC.

The sites affected included the main web sites for the following projects: curl, c-ares, trio, libssh2 and Rockbox. It also affected mailing lists and CVS repositories etc for some of those.

The reason for the outage has been explained by the ISP (Black Internet) to be because of some kind of sabotage. Their explanation given so far (first in Swedish):

Strax efter kl 20 i måndags drabbades Black Internet och Black Internets kunder av ett mycket allvarligt sabotage. Sabotaget gjordes mot flera av våra core-switchar, våra knutpunkter. Detta resulterade i ett mer eller mindre totalt avbrott för oss och våra kunder. Vi har polisanmält händelsen och har ett bra samarbete med dem.

Translated to English (by me) it becomes:

Soon after 8pm on Monday, Black Internet and its customers were struck by a very serious act of sabotage. The sabotage was made against several of our core switches. This resulted in a more or less total disruption of service for us and our customers. We have reported the incident to the police and we have a good cooperation with them.

Do note that you could keep track of this situation by following me on twitter.

It’s good to be back. Let’s hope it’ll take ages until we go away like that again!

Update: according to my sources, someone erased/cleared Black Internet’s core routers and then they learned that they had no working backups so they had to restore everything by hand.

c-ares

c-ares 1.6.0

With a few bug fixes and general improvements, c-ares 1.6.0 was released just now with these new additions:

  • support for the glibc “rotate” resolv.conf option (or ARES_OPT_ROTATE)
  • ares_gethostbyname_file()
  • ares_dup()
  • ares_set_socket_callback()

(the man pages for the new functions are not yet available on the web site but I’m meaning to get to that soonish)

Open Source

c-ares 1.5.3

I’m happy to announce the release of c-ares 1.5.3. c-ares is an asynchronous name resolver and somewhat generic DNS library with a liberal MIT-style license.

The news this time include:

  • fix adig sample application compilation failure on some systems
  • fix pkg-config reporting of private libraries needed for static linking
  • fallback to gettimeofday when monotonic clock is unavailable at run-time
  • ares_gethostbyname() fallback from AAA to A records with CNAME present
  • allow –enable-largefile and –disable-largefile configurations
  • configure process no longer needs nor checks size of curl_off_t
  • library will now be built with _REENTRANT symbol defined if needed
  • Improved configure detection of number of arguments for getservbyport_r
  • Improved query-ID randomness
  • Validate that DNS response address matches the request address
  • fix acountry sample application compilation failure on some systems

I’m also happy to see that the development version of Wireshark is currently using c-ares.

If you’re a graphics person, we’ll appreciate some kind of logo/symbol thing for the project!