djm's scribble

Entries in the Category “code”

Booting OpenBSD on the IO-DATA USL 5P

written by djm, on Apr 12, 2007 10:38:00 PM.

I have been playing with a new toy that was given to me in Japan: an IO-DATA USL 5P. These are a tiny computer, between the size of a deck of cards and a small paperback, which has a Hitachi SH-4 processor and 64MB of RAM. It boots off compact flash and has an Ethernet and five USB ports to talk to the outside world. They ship with a customised Linux distribution, but it is now possible to install OpenBSD on them.

Unfortunately, the installation process is a little fiddly. These devices don’t expose an RS-232 serial port for a console, only some TTL pins on their circuit board that use the wrong voltage to attach to normal serial devices. Nevermind, a $5 MAX3232 chip from Maxim, another $1 worth of capacitors and some breadboard (ok, and some swearing) had me a level converter that allowed me to attach the device to a standard serial port. OpenBSD booted fine, but my first attempts to install were thwarted due to a loose link on my breadboard. Once I fixed that, OpenBSD installed fine (via USB 802.11 no less) and I have a miniscule computer running Unix!

IO-DATA USL 5P and RS232 level converter

AsiaBSDCon 2007

written by djm, on Mar 19, 2007 2:38:00 PM.

I have just returned from AsiaBSDCon 2007 where I presented (PDF slides) a paper on the security measures we have implemented in OpenSSH.

The conference was held in Tokyo, at Tokyo University. The venue was great and the conference was well organised, except that the single track made for very long days (9am to 7:30pm). There were some good papers and presentations at the conference; apart from the OpenBSD ones, I particularly liked Antti Kantee’s “pass to userspace framework filesystem” work, prof. Murai’s wide ranging talk on BSD, the Internet and pervasive networking and the three IPv6 mobility papers from the researchers at IIJ and Keio university.

Tokyo was great fun, as was catching up some some fellow OpenBSD hackers and appreciators (I’ll post some photos once I have sorted them). After the conference I spent a day and a half pounding the streets of Ginza, Shibuya, Harajuku, Shinjuku, Asakusabashi and Akihabara looking for gifts for my wife and son (ok, ok, Akihabara was for me) - this turned out more difficult than I expected, so I ended up covering a lot of ground and getting very sore feet. I didn’t buy much for myself, though Mark from OpenBSD support Japan kindly gave me an IO Data USL-5P to hack on (only slightly larger than a deck of cards!)

I also got to fulfill my childhood dream of visiting a Japanese amusement arcade, but I think I was at least 10 years too late: most of them seem to have been subsumed by Pachinko. Pachinko parlours are mind-altering - if you have ever experienced sensory overload on visiting a casino slot-machine room, start by squaring that level of din and flashing lights then add thick cigarette smoke, the tinkling rattle of hundreds of thousands little balls and epileptic flashing graphics on each machine. msf@ theorised that Pachinko parlours are the reason behind Japan’s fecund production of avant-garde noise artists. (I can’t imagine how Hunter S. Thompson’s masterwork would have turned out had he been given access to Pachinko.) It was too much for me - I didn’t play, sticking instead to the new versions of Virtua Cop, Time Crisis and watching Japanese schoolgirls mow down zombies with submachine guns (a new version of House of the Dead).

I’m already looking forward to going back to Japan again, even though my 日本語 has atropied substantially. One thing I’ll have to do is pick up a W-CDMA phone before I leave as I don’t think I could cope without having my telephony cortex lobotomised out again.

cfsm - finite state machine code generator

written by djm, on Feb 7, 2007 6:24:00 PM.

I have been playing around with replacing or renovating parts of OpenSSH’s sshd server, especially replacing the hand-crafted (and brittle) mainloop with Niels’ libevent. While playing with this on the bus, it occurred to me that several large chunks of the application would be more clear if the state machines they implemented were more explicit. The privilege separation monitor and the channels system (port forwarding, login session, etc.) are good examples of this.

Because there are several places I’d like to do this, I wrote a small code generator to help: cfsm. cfsm takes a simple text state machine specification and outputs a C source and header file and optionally a graphviz dot file for plotting (see below). The code it generates includes transition checking, so it should be impossible to make any transition not expliclty defined. It also allows for the definition of precondition callbacks that muct be satisfied before a transition is allowed. A small example:

state T1a
        initial-state
        next-state T2
	onexit-func do_something
state T1b
        initial-state
        next-state T2
state T2
        next-state T3
	exit-precondition is_bottom_half_of_the_hour
state T3
        next-state T2
        next-state T3
        next-state T4
state T4
	entry-precondition am_i_in_a_good_mood

This produces the following FSM API: (comments abridged)

enum state { T1a = 0, T1b = 1, T2 = 2, T3 = 3, T4 = 4, };
struct fsm;
struct fsm *state_init(enum state initial_state,
    char *errbuf, size_t errlen);          /* Allocate a FSM */
void state_free(struct fsm *fsm);          /* Free a FSM */
int state_advance(struct fsm *fsm, enum state new_state,
    char *errbuf, size_t errlen);          /* Move to a new state */
const char *state_ntop(enum state);        /* State number -> name */
const char *state_ntop_safe(enum state);   /* Same, but doesn't return NULL */
enum state state_current(struct fsm *fsm); /* Returns the current state */

The user of the API will need to provide the callback functions:

/* Preconditions - must return 0 on success */
int is_bottom_half_of_the_hour(enum state current_state, enum state new_state);
int am_i_in_a_good_mood(enum state current_state, enum state new_state);
/* Transition function */
int do_something(enum state current_state, enum state new_state);

In case you are wondering what the graphviz output looks like, here is an example:

cfsm example
(yes, I know I am abusing standard notation)

This is all very basic stuff, but I think that having simple tools like this around saves time (well, when amortised over their lifespans) and reduces mistakes. The code for cfsm is in anonymous CVS (:ext:anoncvs@anoncvs.mindrot.org/cvs via SSH) and is browsable via cvsweb. If anyone is interested I can make a tarball release of it - just let me know via email.

Windows binaries for Python modules

written by djm, on Aug 8, 2006 4:28:00 PM.

After a lot of futzing around with .NET SDKs, environment variables, platform SDKs and Python, I have finally been able to build Windows binaries of most of my Python modules. The path was smoothed greatly by this guide, but it still needed additional tweaking. As a result, py-bcrypt, py-editdist and py-radix now all have Windows binaries available.

The porting problems were a mix of the mundane (missing integer types) and the stupid (no snprintf?, why “winsock2.h” instead of “sys/socket.h”?). Everything works except for IPv6 addresses in py-radix: the Windows getaddrinfo function flatly refuses to parse them. I suspect that if I had an IPv6 stack installed then it would magically work, but IMO that is broken behaviour - parsing numeric addresses should work regardless of what the user happens to have enabled.

py-editdist

written by djm, on Jul 6, 2006 12:31:00 AM.

I just wrote and released py-editdist, a fast CPython module to calculate the Levenshtein edit distance between two strings. Wikipedia has a good description of the actual algorithm, and quite a few sample implementations. I looked at using the Python one from there, but it was too slow - taking around ten milliseconds to calculate the distance between two ~50 character strings, which is a problem because I need to do it about 10 million times per day :) py-editdist does the same calculation in ~70 microseconds, which is in the realm of usable at least…

... and a Java version too

written by djm, on May 24, 2006 4:24:00 PM.

After releasing py-bcrypt, I noticed that there is no equivalent for Java. Worse, the top two Google recommendations for “Java password hashing” and “Java password encryption” are really bad: one example uses reversible encryption (using single-DES ECB mode no less), where reversible encryption should only be used when it is really needed and as a last resort. The others recommend the use of an unsalted hash, which allows reverse lookup of the original password from a pre-computed dictionary of digests (a.k.a a “rainbow table”). (I refuse to link to the articles, because by doing so I would further cement their status at the top of Google’s rankings and thereby perpetuate their bad advice)

Why should developers care about the reversability of password encryption or hashing? Because users have been shown to frequently use the same password in multiple places, and if your password database is disclosed then the attacker can revese it at their leisure and then use it to attack other services that your customers use.

I think the best remedy for this situation is to release an implementation of a good password hashing scheme for Java: jBCrypt. This uses the same algorithm as py-bcrypt: Provos and Mazieres’ “A Future-Adapable Password Scheme”, first introduced by OpenBSD and now available in other operating systems (including Solaris 10+). jBCrypt has a really simple API, a license that is very permissive, and a set of JUnit tests to make sure it works.

Released py-bcrypt

written by djm, on May 22, 2006 10:51:00 PM.

I just released py-bcrypt, a Python implementation of OpenBSD’s Blowfish password hashing after needing a good password hash for a web application. Hopefully making it available it will encourage people to use it instead of perpetuating stupidities like storing their web application’s passwords unhashed, or as a simple md5(plaintext).

OpenSSH 4.4 testing

written by djm, on May 9, 2006 10:51:00 AM.

We are getting close to releasing OpenSSH 4.4 and this version will include a surprising number of useful features. There is some of a list in the call for testing email that I sent to the portable OpenSSH list a week ago. If you use OpenSSH, then please try one of the snapshot releases, or at least download it, run “./configure && make tests”. The snapshot releases are very stable, and you could consider running them on workstations or non-production servers. You can report your findings (success or failure) back to the openssh-unix-dev@mindrot.org mailing list (if you are not subscribed then you will need to complete a one-time authentication challenge to prove that you are not a spam robot).