Tag Archives: libssh2

libssh2.haxx.se

libssh2I've played around with a possible new design for the libssh2 web site and I've put it up on libssh2.haxx.se for everyone to play around with and comment (on the libssh2-devel list please).

The original and actual home page for the project is still over at www.libssh2.org but I'm not happy with that because of a few things:

The wiki duplicates info that we these days write in man pages and hardly anyone updates the wiki so it lags behind or just contains false or outdated info. I also think having the entire site a wiki a bit problematic for things like menus and generic site layout etc.

I want daily snapshots, web versions of the man pages and mailing list archives to be somewhat integrated in the site to be easier to find.

I already hosted libssh2-related stuff anyway so I've just packaged it in a slightly more friendly way. In my view.

What do you think?

libssh2 upped a notch

There have been some well-founded criticism against libssh2 for a long time for its bad transfer performance when doing SCP and SFTP based transfers. Tests have proved it to be significantly slower than the openssh based alternatives in comparisons done in similar conditions. We're talking down to a tenth(!) of the speed for SFTP.

Luckily I have a unnamed (by agreement) sponsor who pays me for improving this.

Giving it some love

I basically started out reading the SFTP code and cleaned it up as I went over it, and I added some clarifying comments etc. I found some irregularities that I fixed. Soon I could spot an obvious performance boost, like perhaps 3-4 times the previous speed. But since SFTP was painfully slow originally, this was still very crappy compared to openssh.

I then switched over to plain SCP tests. SCP is basically just an "scp" command sent over SSH and then streaming the data over a plain SSH "channel", while SFTP is a whole additional protocol layer on top. Thus SCP is more low-level, on the actual SSH level, and the foundation on which SFTP runs anyway so getting SCP faster was fundamental.

Make it speedier

My initial tests with libssh2 1.0 showed libssh2 to download data at roughly 25% of the speed of openssh when SCPing a 1GB file from an openssh server running on localhost. The openssh client shows roughly 40MB/sec on my test box.

Also, just checking my CPU load meter while doing the libssh2 transfers showed that it certainly wasn't hitting the roof or anything. It was barely even noticeable! Of course something was really wrong but what was it?

SSH has a lower protocol layer that does the entire encryption thing, the transport layer, but on top of that is the "channel layer" that is packet based for sending data back and forth over the transport layer. This channel thing has a receive window concept, much like TCP itself has, which tells the remote side how much data it is allowed to send until it gets further notice.

libssh2 1.0 had a very conservative windowing logic. It started with a default window size of 64KB and it upped it at every read with the same amount that was read (which then could be 1K to 16KB something depending on the app).

My remake of this was to simplify the logic, read data from the network more evenly distributed over time, update the window size much less frequent and increase the window size by magnitudes! I found that when using a window size of 38MB (600 times the previous default size!!) things started flying.

Improved

With these modifications, libssh2 transfers SCP at close to 40MB/sec! SFTP is still left behind at a "mere" 14MB/sec on the same test setup but it has its own set of problems and solutions. Now this discussion on the libssh2 list is more about how to sensibly size the window to work the best way for different situations.

SFTP is a protocol that works more on file operations. The client sends OPEN, READ and CLOSE requests and the server replies with status and data. The READ request asks for N bytes starting at offset Z so a simple implementation like libssh2 asks for chunk after chunk in a serial manner, increasing the offset as it loops over the range. This causes a back-and-forth effect that certainly does not make optimized use of the network bandwidth.

SFTP ping pong

openssh has a nifty approach to enhance throughput for SFTP: it sends off and handles multiple outstanding READ requests in parallel so that it better can keep things busy (and the reverse when doing uploads). That concept is slightly harder to do with an API like the one libssh2 offers but it is of course still quite doable. I suspect that we might achieve results somewhat faster by simply use multiple connections as then we can remain using this simplistic approach but still use the full bandwidth. (Yes, I realize multiple connections may not be feasible for all applications.)

Previous tests we've done with SFTP uploads using multiple connections have proven libssh2 to be on par or even better than competitors on both Windows and Mac.

Please test

I'll leave it like this for now. I'll be very happy if people could test this version and report findings so that we make sure this is working and stable enough to release soonish. We'll need to do something that offers window size controlling to apps, but we'll discuss that further on the mailing list. Join in!

logo1-250

A new year with new fun

I had a great and relaxing winter/Christmas holiday and hence my silence here.

I'm now back up to speed, with a podcast interview done yesterday (I'll post another entry when it gets available), I do some funded development on libcurl and libssh2 in the background while I'm spending my days at my client's place working on a 10G traffic analyzer product.

It was rather calm during "the break" but I've now noticed that at least the curl project has gotten significantly increased activity again. We're on a feature freeze now for the January release, but there seems to be at least 4 patches pending adding new stuff for the release planned to come after this (around March if things go well).

libssh2 1.0!

I'm happy to say that I've just uploaded the 1.0 release of libssh2 to sourceforge! (I must confess that I strongly dislike the "file release" thing of sourceforge but libssh2 has always been released there so I'm just continuing the tradition really...)

The changes can be read closer in the package but the main things are these:

Added libssh2_sftp_tell64()

Added libssh2_session_block_directions()

Added libssh2_channel_request_pty_size_ex()

Added libssh2_sftp_seek64()

Added the beginning of a test suite

Deprecated libssh2_base64_decode()

Fixed many bugs - possibly the biggest item really since several of the fixed bugs were of the kind that prevented the lib to do successful transfers in many cases.

This is the primary library for SSH-based communication that I know of. Note that this is not the same project as the libssh one. I once did a thorough comparison with all SSH libraries I could find, and libssh2 was then the one nearest to my feature requirements. We have since then taken it a lot further and it is now a fairly stable and good library for SSH-based transfers and generic communication.

There's of course still (a lot of) things left to do, but here's the 1.0 as a sign that this is now a lib ready to get used!

Projects in need of your help

I'm involved in numerous projects, and a subset of them take a lot of my "copious" spare time. This has the unfortunate downside that a few other projects get left behind a bit. Projects that also really could use with some more attention and improvements. Two of the most obvious examples of this are c-ares and libssh2. Coincidentally, both of these projects are also used by libcurl (although of course also by others).

c-ares is a library that performs asynchronous DNS lookups. It is quite mature and functional already, as it is based on the ares project and has been proved in use for quite some time. There are currently one or two issues that have appeared recently when the Debian project tried to provide the curl package built with c-ares...

libssh2 is a client library for talking SSH2 with servers. There are actually not very many SSH libraries "out there", and in an evaluation I did a few years ago libssh2 was the best one around. libcurl uses libssh2 for SCP and SFTP transfers, and it (libssh2) does suffer from a few API flaws, a few bugs and perhaps most noticably it is significantly slower than the openssh tools in just about all transfer tests.

I'm still highly involved in both of these projects, but lack of time prevents me from participating as much as I'd like to.