Category Archives: libssh2

Development, libssh2, Open Source

libssh2 vs libssh

There are only two open source libraries for SSH that I am aware of. At least that are at the fundamental layer, written in C.

I researched the SSH library market years ago when I stuck with libssh2 as the one I thought was most promising, and since then I and others have taken it much further. The lib that I didn’t go with at that time, confusingly enough named libssh, recently came out with a new release.

Since there is now clearly two active open source SSH libraries it feels like we should help our users and potential newcomers by explaining how our projects and libraries differ. As a little teaser: one of the libraries turned out more than twice as fast as the other in my test…

While I admit to not having actually used libssh for real, I’ve read the docs and I’ve tried it a little bit. My take at a comparison is now online at:

http://www.libssh2.org/libssh2-vs-libssh.html

I will highly appreciate your feedback and additional things that differ between the two! The list isn’t really much to boast about as it currently looks!

cURL and libcurl, libssh2

Adding known hosts support

… to libcurl and libssh2!

I’m about to start this little mini adventure, so if you’re one of the guys out there who’s been looking forward to be able to do even more (Open)SSH-like things with curl and libcurl when we use SCP and SFTP then consider this a little notification to start listening!

This will require improvements and changes in both projects, and funnily enough I’m already involved knee-deep in both so that shouldn’t cause any problems. I do however greatly appreciate feedback and reviews of my pending implementation proposals! I want this done in a way that benefits many and that isn’t too likely to break at least within the nearest future.

Ok, enough of that. Stand by for posts to the mailing lists. I’ll start off with the libcurl one which will thus be a slightly higher level API for all this. I’ll update this blog post later on to feature direct links to my proposals. Please consider posting responses to the suggestions to the appropriate mailing list!

The libcurl proposal

The first mail to libssh2-devel

libssh2

libssh2 1.1

I’m happy to announce that we now have a version 1.1 of libssh2 released! Noticeable changes this time include:

  • Downloads using SCP or SFTP are now significantly faster
  • Added a Libtool -export-symbols-regex flag to reduce the number of exported symbols in shared libraries.
  • Added a bunch of new man pages and renamed some of the previous ones
  • Enhanced download performance
  • Made libssh2_scp_recv() and libssh2_scp_send() deal with spaces in filenames
  • Fixed the bad randomness and off-by-one in libssh2_channel_x11_req_ex()
  • Added libssh2_version()
  • Fixed libssh2_channel_direct_tcpip_ex() to not fail when called a second time
  • Fixed libssh2_channel_write_ex problems in blocking situations
  • ‘make check’ runs fine on cygwin
  • Added libssh2_channel_receive_window_adjust2() and deprecated  libssh2_channel_receive_window_adjust()
  • better socket error handling internally on win32
  • libssh2 now always set the socket non-blocking internally and deals with the interface as blocking or non-blocking set by libssh2_session_set_blocking.

The library is rapidly maturing and is getting really usable. I’m happy to see that there’s a community slowly building up around this and I’m also grateful for my sponsor paying for parts of the fixes that contribute to make this release the best ever in libssh2’s history.

libssh2

libssh2

libssh2.haxx.se

libssh2I’ve played around with a possible new design for the libssh2 web site and I’ve put it up on libssh2.haxx.se for everyone to play around with and comment (on the libssh2-devel list please).

The original and actual home page for the project is still over at www.libssh2.org but I’m not happy with that because of a few things:

The wiki duplicates info that we these days write in man pages and hardly anyone updates the wiki so it lags behind or just contains false or outdated info. I also think having the entire site a wiki a bit problematic for things like menus and generic site layout etc.

I want daily snapshots, web versions of the man pages and mailing list archives to be somewhat integrated in the site to be easier to find.

I already hosted libssh2-related stuff anyway so I’ve just packaged it in a slightly more friendly way. In my view.

What do you think?

libssh2, Network

libssh2 upped a notch

1 Comment

There have been some well-founded criticism against libssh2 for a long time for its bad transfer performance when doing SCP and SFTP based transfers. Tests have proved it to be significantly slower than the openssh based alternatives in comparisons done in similar conditions. We’re talking down to a tenth(!) of the speed for SFTP.

Luckily I have a unnamed (by agreement) sponsor who pays me for improving this.

Giving it some love

I basically started out reading the SFTP code and cleaned it up as I went over it, and I added some clarifying comments etc. I found some irregularities that I fixed. Soon I could spot an obvious performance boost, like perhaps 3-4 times the previous speed. But since SFTP was painfully slow originally, this was still very crappy compared to openssh.

I then switched over to plain SCP tests. SCP is basically just an “scp” command sent over SSH and then streaming the data over a plain SSH “channel”, while SFTP is a whole additional protocol layer on top. Thus SCP is more low-level, on the actual SSH level, and the foundation on which SFTP runs anyway so getting SCP faster was fundamental.

Make it speedier

My initial tests with libssh2 1.0 showed libssh2 to download data at roughly 25% of the speed of openssh when SCPing a 1GB file from an openssh server running on localhost. The openssh client shows roughly 40MB/sec on my test box.

Also, just checking my CPU load meter while doing the libssh2 transfers showed that it certainly wasn’t hitting the roof or anything. It was barely even noticeable! Of course something was really wrong but what was it?

SSH has a lower protocol layer that does the entire encryption thing, the transport layer, but on top of that is the “channel layer” that is packet based for sending data back and forth over the transport layer. This channel thing has a receive window concept, much like TCP itself has, which tells the remote side how much data it is allowed to send until it gets further notice.

libssh2 1.0 had a very conservative windowing logic. It started with a default window size of 64KB and it upped it at every read with the same amount that was read (which then could be 1K to 16KB something depending on the app).

My remake of this was to simplify the logic, read data from the network more evenly distributed over time, update the window size much less frequent and increase the window size by magnitudes! I found that when using a window size of 38MB (600 times the previous default size!!) things started flying.

Improved

With these modifications, libssh2 transfers SCP at close to 40MB/sec! SFTP is still left behind at a “mere” 14MB/sec on the same test setup but it has its own set of problems and solutions. Now this discussion on the libssh2 list is more about how to sensibly size the window to work the best way for different situations.

SFTP is a protocol that works more on file operations. The client sends OPEN, READ and CLOSE requests and the server replies with status and data. The READ request asks for N bytes starting at offset Z so a simple implementation like libssh2 asks for chunk after chunk in a serial manner, increasing the offset as it loops over the range. This causes a back-and-forth effect that certainly does not make optimized use of the network bandwidth.

SFTP ping pong

openssh has a nifty approach to enhance throughput for SFTP: it sends off and handles multiple outstanding READ requests in parallel so that it better can keep things busy (and the reverse when doing uploads). That concept is slightly harder to do with an API like the one libssh2 offers but it is of course still quite doable. I suspect that we might achieve results somewhat faster by simply use multiple connections as then we can remain using this simplistic approach but still use the full bandwidth. (Yes, I realize multiple connections may not be feasible for all applications.)

Previous tests we’ve done with SFTP uploads using multiple connections have proven libssh2 to be on par or even better than competitors on both Windows and Mac.

Please test

I’ll leave it like this for now. I’ll be very happy if people could test this version and report findings so that we make sure this is working and stable enough to release soonish. We’ll need to do something that offers window size controlling to apps, but we’ll discuss that further on the mailing list. Join in!

logo1-250

cURL and libcurl, libssh2

A new year with new fun

2 Comments

I had a great and relaxing winter/Christmas holiday and hence my silence here.

I’m now back up to speed, with a podcast interview done yesterday (I’ll post another entry when it gets available), I do some funded development on libcurl and libssh2 in the background while I’m spending my days at my client’s place working on a 10G traffic analyzer product.

It was rather calm during “the break” but I’ve now noticed that at least the curl project has gotten significantly increased activity again. We’re on a feature freeze now for the January release, but there seems to be at least 4 patches pending adding new stuff for the release planned to come after this (around March if things go well).

libssh2

libssh2 1.0!

I’m happy to say that I’ve just uploaded the 1.0 release of libssh2 to sourceforge! (I must confess that I strongly dislike the “file release” thing of sourceforge but libssh2 has always been released there so I’m just continuing the tradition really…)

The changes can be read closer in the package but the main things are these:

Added libssh2_sftp_tell64()

Added libssh2_session_block_directions()

Added libssh2_channel_request_pty_size_ex()

Added libssh2_sftp_seek64()

Added the beginning of a test suite

Deprecated libssh2_base64_decode()

Fixed many bugs – possibly the biggest item really since several of the fixed bugs were of the kind that prevented the lib to do successful transfers in many cases.

This is the primary library for SSH-based communication that I know of. Note that this is not the same project as the libssh one. I once did a thorough comparison with all SSH libraries I could find, and libssh2 was then the one nearest to my feature requirements. We have since then taken it a lot further and it is now a fairly stable and good library for SSH-based transfers and generic communication.

There’s of course still (a lot of) things left to do, but here’s the 1.0 as a sign that this is now a lib ready to get used!

cURL and libcurl, libssh2, Open Source

Projects in need of your help

I’m involved in numerous projects, and a subset of them take a lot of my “copious” spare time. This has the unfortunate downside that a few other projects get left behind a bit. Projects that also really could use with some more attention and improvements. Two of the most obvious examples of this are c-ares and libssh2. Coincidentally, both of these projects are also used by libcurl (although of course also by others).

c-ares is a library that performs asynchronous DNS lookups. It is quite mature and functional already, as it is based on the ares project and has been proved in use for quite some time. There are currently one or two issues that have appeared recently when the Debian project tried to provide the curl package built with c-ares…

libssh2 is a client library for talking SSH2 with servers. There are actually not very many SSH libraries “out there”, and in an evaluation I did a few years ago libssh2 was the best one around. libcurl uses libssh2 for SCP and SFTP transfers, and it (libssh2) does suffer from a few API flaws, a few bugs and perhaps most noticably it is significantly slower than the openssh tools in just about all transfer tests.

I’m still highly involved in both of these projects, but lack of time prevents me from participating as much as I’d like to.

libssh2

libssh2 0.18

I stole some time from my family this weekend and I managed to put together and released a fresh libssh2 tarball, labeled 0.18.

I’m not really too fond of spending a lot of time with a sub-zero version number, but the recent releases have felt like just minor improvements to the previous so bumping up to 1.0 has a certain mental brake all over it. I guess I need to just ignore the brake and take the plunge soon, as I believe it’ll make users more likely to start actually using the lib and that will be good.