A story
A few years ago, at Danger, I wrote an SSH client for the Hiptop (or Sidekick) phone, which we gave away to developers for free as a demonstration of our API and the cool things you could do with it. Later, the developer program would be effectively shut down, and the app would be renamed “Terminal Client” and sold to hapless users for $10. But I didn’t know that yet.
The protocol implementation was awful. SSH protocol “documentation” was just a handful of poorly written RFC drafts that described things in seemingly random order. I hacked on some java code until I got it mostly functioning and left it at that. But it was so buggy and bare-bones that it often would only connect to certain OpenSSH daemon versions, and this eventually bothered me enough that I wanted to fix it.
Python was my favorite language at the time. The syntax is so simple and clean that I feel like I’m writing in pseudocode. It’s a very close match to “the way I think” when I’m trying to solve a problem, so there’s a very low impedance between what I think and what I type. If I’m trying to sketch out a rough draft of code, I usually do it in python as a prototype. And since the SSH protocol seemed like a difficult problem to me, I decided I should try to write an SSH library in python, as a prototype for doing it in java.
The basic idea was to use a thread to handle the protocol level of the socket,
so that SSH’s packet windowing requests would get immediate responses, and
supply an API to the client that looked just like a normal python file. Data
should arrive, get handled by the protocol thread, and stuffed into a buffer
so that a client read could get it. The read would trigger window feedback
to the server, but the client could go do other stuff while waiting for data.
It worked beautifully. I called it paramiko as a dorky esperanto pun and released it, and after a while, to my moderate surprise, people were using it. I didn’t really anticipate that it would be useful for automating tasks for working with server machines and clusters, though it’s kind of obvious in retrospect.
Python’s thread model (the dreaded Global Interpreter Lock) isn’t awesome, but when threads are mostly blocked on I/O, they’re good enough. The bazaar team obsessed about performance, and helped find and solve some real bottlenecks, so that in the end, paramiko was slower than OpenSSH, but not by much.
Over time, the feature set exploded. Now it can be an SSH client or server, or an SFTP client or server, and has a few experimental SFTP features and a few made-up ones (just to see if they were useful). I wanted to try writing a standalone SFTP server based on it, for making little file-server sandboxes, but… I only have so much free time. :)
I did end up using what I learned to write “jaramiko”, a java library for SSH, and used that in later releases of the Sidekick SSH client, but it was never really as powerful or successful as paramiko. In retrospect, it’s obvious why: Java is not even remotely as expressive or concise as python, so most of the java code is boilerplate that obscures the real logic. It’s hard to keep up-to-date and maintain, much less add new features to.
What now?
I haven’t made a new release of paramiko since July. It feels like even longer than that. Really, to me, the library is finished. I’d like to fix any remaining bugs in it, but I don’t get many bug reports any more, and most of the mailing list support questions are answered by other users.
I don’t want the project to die. I just have to admit to myself that I’m not really taking a leading role in it anymore. So I’ve migrated the repository to git and put it on github here: github.com/robey/paramiko Anyone can fork it there and submit patches, and I’ll make regular releases as long as there are still bugs to fix, or new features that look cool.
Why git? I like bazaar better (see some previous rants on this blog) but the rest of the world has chosen git, and I don’t have the energy to keep fighting that fight. It’s also easier for me if I can use the same VCS everywhere. Keeping the project on github, in front of my face, will make it less likely to bitrot. At least, that’s my hope.