Category Archives: Development

Development, Open Source, Technology, Web

My HTC Magic Review

This is my first “smartphone” I’ve owned myself so of course I have nothing else this fancy to actually compare against. I’ve played around with others’ a few times but that doesn’t really count. I’ve owned perhaps 8 mobile phones since I got my first one 1996, and they have all been Nokias and Sony Ericssons.

I was never really interested in iPhone due to many reasons. It is not open. It has a (very) restricted app distribution mechanism. It forbids apps from running simultaneously etc. And it has a pretty strong connection with itunes with no proper mass-storage syncing supported. But I admit that it has a slick UI and many cool apps.

My plan is to get some Android hacking going eventually and this is basically the first Android phone that has reached Swedish soil. I mean without requiring me to bend over backwards to get it, as I’m sure I could’ve bought previous Android phones from obroad if I really wanted to.

Random good things:

  • it’s fast, most things run faster than on my previous Sony Ericsson thing and yet this is way more advanced with much bigger screen estate and fancier UI
  • it has a nice gui that you mostly can guess how to work with
  • I love being able to use a qwerty-style keyboard when messaging instead of relying on T9 etc
  • wifi is fun, but with a decent data plan it basically only brings me slightly improved speed and I often can’t even tell the difference!
  • the integration with the Google services are nice, gmail and maps most noticeably
  • there really are a bunch of existing cool apps (I know iphone has lots more, but there are still thousands)
  • it has a much better approach to messaging, similar to what I’ve seen in the iphone, than I’ve ever experienced in a Nokia or Sony Ericsson. It focuses on conversations and keeps the “thread”.HTC Magic
  • I really really like the feeling of it being a networked thing that also can make phone calls. I can browse, use maps, use gmail just as easily as I can message or call people. With my previous phones all the internet-related services always felt tacked on like a very late afterthought.
  • The notification system is nice, and the three-screen wide “home” with its widget-system is really neat.

Bad stuff:

  • I’ve had some apps crash on me on occasion. But it’s rarely a problem as they’re restarted automatically for me.
  • Toggling wifi on/off a lot can sometimes lead to me not getting any data network at all, and I’ve had to reboot the phone to get back to phone-based (Edge/3G) data.

On-screen keyboard

Of course any and all geek friend I have ask me about how I deal with the on-screen keyboard. I must admit I’m still quite fond of it. Mostly because a physical keyboard makes the phone clonky and it adds physical contraints and wear-points that I don’t like. So the keyboard is a bit small, especially when the phone is in portrait mode, but the suggested completions are fine and I believe I’m already typing pretty quickly on the thing. When I ssh’ed from the phone to one of my servers I did find the obvious lack of cursor keys (to for example navigate an ordinary ncurses-based app or the command line history of a bash prompt) but other than that I really can’t complain.

Background Applications

One obvious advantage compared to iphones is of course the ability to run applications exactly the way I’d like. I can actually run the irc client and then have it in the background while I go browse the web or answer a call or whatever and then at my choice go back to the still connected irc client. In fact when playing with this it feels like a really ridiculous restriction of the iphone.

Comparing to my SE w550i

My previous phone is 94 grams compared to the Magic’s 116. The magic has a much bigger screen. The magic is roughly 11mm wider and 14mm taller. That makes it use 30% more volume (85 cm2) but still fits fine in the front pocket of any set of pants I use. The magic claims a lot longer battery life, but given that it has so much functionality I can’t help to play with all the time I doubt it’ll notice. It’ll more likely run down fast simply because I’ll use it more.

I’m also pleased that there’s no problem to just plug in the Magic to my Linux desktop and copy/sync the photos and the videos etc.

Google Integration

I realize some people will feel that the very tight integration with Google and Google’s services is a downside as it adds just another item that Google “owns” in your life. Still, it makes the experience very slick and as a user I get a lot of stuff “for free” as it just connects to lots of things that I already used and had accounts on. So gmail, sharing photos on picasaweb etc “just works”.

Development, Linux, Rockbox, Technology

Decrypting ipods

Recently we’ve seen progress by the linux4nano guys in their quest to get custom code to run on an Ipod Nano 2nd generation. They’ve apparently managed to extract the bootrom off a 2nd gen ipod nano (my copy of their extracted data is here – a reminder on objdump usage: “arm-elf-objdump -D --target binary -marm [file]“). I believe their intent is to port Linux to the newer ipods. Possibly ipodlinux. They do mention providing the necessary info to Rockbox and yes we will welcome it.

A large crowd of Rockbox hackers have joined their IRC channel and have been hanging out with them and helped out discussing ideas and pushed them towards publishing their news and infos on how this all is accomplished etc. Their SVN repo hosts some (most?) of the tools made so far.

The Rockbox wiki page for nano2g has been updated and hopefully it will keep track of what happens.

There have been speculations, but I don’t yet know based on what facts, that this recent news and hacks will be usable on other recent (encrypted) ipod models.

Summary: very interesting progress has been made. Lots of it is still left to figure out. There seems to be a bunch of skilled people around and now we’re seeing information and documentation for this getting published so I can’t but to hope for a bright future!

Development, Electronics, Rockbox, Web, Windows

Concepts of a new distributed build

It was time to make an overhaul of our distributed builds system for Rockbox. The one currently in place is quite fancy and it does build 106 builds in around 7-8 minutes, but during the years it has served us we have found a few areas where we want to improve.

The goals for the new system were primarily:

  • do all the builds faster
  • reverse the connection so that people can contribute clients easier
  • make a system that is more allowing for slower machines to contribute

The biggest weaknesses of the existing system:

  • The master uses ssh to the distributed clients, which forces them to have an accessible ssh server and port etc. It also makes it awkward for people behind NATs who wants to run more clients.
  • It only hands out a particular build to one client, so thus if a large build happens to get handed to a slow client towards the end of a build round, all the other clients will sit idle waiting for the last client to finish.
  • The build and the subsequent upload of results to the master are synchronous, so thus a client with a very slow uplink may spend a significant time on the upload before it can start the next build.

The  new system is currently in development. It consists of a server that runs on one of our main servers, and there’s a client script that each volunteer contributor runs on their systems.

The clients connect to the master on a dedicated TCP port, specifying user name, password, name of the particular client instance, what particular architectures the client can build and how many bogomips the client boasts. While bogomips is a bogus way to measure anything, we’ve started out using it for a rough way to sort the the build clients based on speed.

The clients keep connected to the server all the time. There’s a ping message from the master every N second of idleness to make sure the connection is kept alive. As soon as the master wants the client to do a build, it sends a message to it detailing exactly how it should build it and using what SVN revision. The client will then do the build at once, upload the results using HTTP to a dedicated place and then tell the server the build is complete.

The server knows about all builds to do at a  commit, what we call a build round. It has a rough “score” or “weight” for each build that grades them in a slow to fast order. When a build round starts, the server will first sort all builds based on number of times they’ve been handed out and as secondary sort key the “weight” of it. Then it loops over the currently connected build clients and hand out builds from the sorted build table. The server then continues to do that until all clients have three builds each to build. As soon as a build is reported to have been completed by a client, that client will get the next build from the sorted build list.

If a client connects to the server and the server deems the client to be too old (since it does specify its version in the handshake message), it will be told to update to a specific version instead and come back then. This way the server can update all build clients when important things are fixed.

The clients will soon start to get assigned builds that already have been assigned to another client. This is not a problem but in fact our intention. The client that completes the build first will simply tell the server, and the server will then tell all the other clients that build that same build that they should cancel that particular build.

A client that joins the server in the middle of a build round will simply get a bunch of builds immediately and join in. A client that disconnects during a build round simply won’t complete its builds and other clients will instead do them. The system is also tolerant against the fact that bogomips is lame to compare computers with, and that the build “score” may not be very accurate or even that some server will have very slow or very fast upload speeds at unpredictable times.

The build master itself does not know when to start a new build round. It simply knows about the concept and it knows how to tell clients to complete a round. To make the master to start a new round, you need to connect to the server’s listening port and issue a special command and provide a password and then you can tell the server to start a build of a specific SVN revision. Or to queue up a build to be performed after the current one if there happens to be one in progress already.

When a full build round is complete, a hundred or so builds have been done, and full packages and log files are now in a directory on the build server, the server will simply trigger an external script that then takes care of updating our build table etc. In fact, every single completed build will optionally trigger an external script to allow web pages or stats pages to get updated as we go.

This build system is currently pretty Rockbox-specific as this is the project and development system we’re writing this for, but there’s really nothing in this that must be this way. I’m sure that if someone (you?) wants to adapt this for another project, I’d be more than happy to assist and to help ensuring that this becomes a more generic distributed build system. Just raise your hand and step forward!

At the time of this writing, (primarily) me and Björn are still ironing out quirks in this new system to hopefully get it going live real soon…

Rockbox

Development, libssh2, Open Source

libssh2 vs libssh

There are only two open source libraries for SSH that I am aware of. At least that are at the fundamental layer, written in C.

I researched the SSH library market years ago when I stuck with libssh2 as the one I thought was most promising, and since then I and others have taken it much further. The lib that I didn’t go with at that time, confusingly enough named libssh, recently came out with a new release.

Since there is now clearly two active open source SSH libraries it feels like we should help our users and potential newcomers by explaining how our projects and libraries differ. As a little teaser: one of the libraries turned out more than twice as fast as the other in my test…

While I admit to not having actually used libssh for real, I’ve read the docs and I’ve tried it a little bit. My take at a comparison is now online at:

http://www.libssh2.org/libssh2-vs-libssh.html

I will highly appreciate your feedback and additional things that differ between the two! The list isn’t really much to boast about as it currently looks!

cURL and libcurl, Development

Dear Apple Inc

12 Comments

Dear Apple Inc,

As one of the primary authors of libcurl and curl, two parts that are included in every Mac OS X release since years back, I was only wondering if you would consider sponsoring me with a Mac, to make it easier for me to do (lib)curl development, tuning and bug-fixing on/for the Mac?green-apple

I really don’t have any particular income from Macs so I don’t see how I can personally motivate spending some 2000 USD on a Mac only for curl. And to be honest, I can’t think of any other reason to get a Mac either!

I did look around Apple’s web site to find an email adress of someone to send my plea to, but I failed. So I’ll just put it here. I have exactly no hope in actually accomplishing anything with this other than putting some attention on how things are.

This post was triggered by recent libcurl bugs that seem to show up only on Mac!

Development, Open Source

Video: How to build Rockbox

I recorded a 10 minute screencast on how to build Rockbox from SVN (on Linux) just now. It’s something I’ve been meaning to do for a long time since I believe a lot of users need to get this shown rather than just a static text describing it. It really is easy and I think my video shows that.

rockboxbuild

The video is available as:

This recording was done using recordmydesktop. I did resize down my browser and reduce the font size to make it look decent in the smaller window. I consider this shot a bit of a test on how it works, and what I can do with it.

Development, Electronics

USB converter woes

1 Comment

USB to rs232 converters are just never sold properly advertising what chip’s inside and right now I want to know if this one UART I’m working with perhaps is not playing fine with my existing converter cable.

I have this XScale PXA270 on a toradex-colibriboard, and it has only one full featured RS232 (FFUART) and I’m about to move things over to the lesser featured BTUART.

A theory is that my current USB converter that is based on a “Prolific PL2303” doesn’t play nicely on the serial port that isn’t a full RS232.

So I ran off and bought a new cable. I grabbed the only model I found in my local Kjell & Company store – it’s quite different looking than my existing but there’s no hint anywhere on the package or inside of it that says what chipset that empowers it.

A quick drive back home (I’m working from home in this assignment), I plugged it in and I got to see this depressingly familiar dmesg output:

usbcore: registered new interface driver usbserial
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial Driver core
usbserial: USB Serial support registered for pl2303
pl2303 2-2.4:1.0: pl2303 converter detected
usb 2-2.4: pl2303 converter now attached to ttyUSB0
usbcore: registered new interface driver pl2303
pl2303: Prolific PL2303 USB to serial adaptor driver

So what now? I hate how (my) computers these days don’t have serial ports while the entire embedded world still very much uses them. I think I’ll go searching in my closet to see if I can find an old crap computer with a serial port to try.

Another theory is that the port simply is broken hw-wise on the dev board but that’s harder to check for me right now.

Update: it was (as usual) only my stupidity that prevented this from working. If I switch it over to the correct baudrate the usb converter does fine. But before I found that out, I did find a computer with a serial port and I did see it working on that too…

Development, Open Source

C Code Commandments

3 Comments

I’m an old school C programmers guy and I stay true to some of the older and commonly used rules present in many open source and similar projects. Since I sometimes rant about this to people, I thought I’d amuse my surrounding by stating them here for public use/ridicule. Of course heavily inspired by the great and superior The Ten Commandments for C Programmers. My commandments are not necessarily in any prio order.

Thy Code Shall Be Narrow

Only in very rare situations should code be allowed to be wider than 80 columns. I want my two or three windows next to each other horizontally and still see the code fine. Not to mention the occasional loading up in an editor in a 80 columns terminal and that is should be possibly print nicely (for reviews etc). Wide code is also harder to read I think, quite similarly to how very wide texts in web pages etc aren’t kind to your eyes either.

Thou Shall Not Use Long Symbol Names

To be able to keep the code easily readable by human eyes so that you quickly get an overview and understand things, you simply need to keep the function and variable names fairly short. Not to mention that the code gets harder to keep within 80 columns if you use ridiculously long names.

Comments Shall Be Plenty

Yes, this is something we know everyone says and few live up to. In statistical analyzes of my own C code I usually reach around 25-27% comments and I’m usually happy with that amount. Comments should explain what is otherwise not obvious in the code.

No Hiding What’s Really Happening

I’m not a fan of overloaded operators or snazzy macros that do fancy stuff without it being noticeable in the code. It should be clear when reading the code what it does. That’s also one of the reasons you don’t catch me doing a lot of C++ work…

Thou Shalt Hunt Down and Kill Compiler Warnings

Compiler warnings may be significant and in some cases they are not. Either way, it is our duty to silence them at all times. Firstly because it is often simpler to fix the code to not warn than to figure out if the warning is indeed right or not, but perhaps primarily because it makes it harder to see new warnings appearing if the old ones have been left there.

Write Portable Code Unless Forced by Evil

You may first believe that your code will live on forever on this single platform with this single compiler, but soon and very soon you will learn otherwise. Then you will cheer this rule as it makes you consider unaligned memory accesses, assuming byte-order of binary data or the size of your ‘long’ variable type.

Repeat Not, Use Functions

I see a lot of “copy and paste” programming in my daily life and I’ve learned that sooner or later such practices lead to sorrow. If you paste the same code on multiple places it not only makes it repetitive and boring to update it when an API or something changes, more seriously it increases the risk that you address bugs only on one out of many places or that the fix differ etc. It also makes the code larger and thus harder to follow and understand.

Thou Shalt Not Typedef Away Pointers

A really nasty habit to be seen in some source codes is when people use typedefs to define their own types that is simply a pointer to something. Like with ‘typedef struct whatever * whatever_t’. While I’m in general against excessive typedefing, I’m fine with them in many cases but not when used to hide pointers to look like “ordinary” types. It makes code harder to follow.

Defines, no fixed numbers

Code that relies on zero and non-zero can get away without this, but as soon as you start relying on more numbers in the code you must start using #defines or possibly enums to make them appear with names in the code. Using names is more clever than hardcoded numbers since you can avoid having to explain the number in a comment, and of course it’ll be easier to change the actual number in the code at a later point without it being a painful search-and-replace operation.