djm's scribble

I just stumbled across this paper (via Charlie Stross’ weblog): A False Sense of Insecurity by John Mueller. Here is a taste:

Even with the September 11 attacks included in the count, the number of Americans killed by international terrorism since the late 1960s (which is when the State Department began counting) is about the same as the number of Americans killed over the same period by lightning, accident-causing deer, or severe allergic reaction to peanuts.

(the “War on accident-causing deer” just doesn’t have the same ring to it). This quote slightly misrepresents the paper as whimsical, it is not - it is a very sober assessment of terrorist threats and appropriate policy responses. Like other such comparisons, I expect this to to be completely ignored by our leaders. The Power of Nightmares it just too tempting for those who want to retain control.

For the last week, I have been receiving spam emails with titles such as “eigenstate is?” and “What?? biconnected?”. Unless these are targetted, I can’t possibly imagine what audience would respond to them.

After a lot of futzing around with .NET SDKs, environment variables, platform SDKs and Python, I have finally been able to build Windows binaries of most of my Python modules. The path was smoothed greatly by this guide, but it still needed additional tweaking. As a result, py-bcrypt, py-editdist and py-radix now all have Windows binaries available.

The porting problems were a mix of the mundane (missing integer types) and the stupid (no snprintf?, why “winsock2.h” instead of “sys/socket.h”?). Everything works except for IPv6 addresses in py-radix: the Windows getaddrinfo function flatly refuses to parse them. I suspect that if I had an IPv6 stack installed then it would magically work, but IMO that is broken behaviour - parsing numeric addresses should work regardless of what the user happens to have enabled.

I just finished reading Scott Aaronson’s NP-complete Problems and Physical Reality (also known as quant-ph/0502072). With excellent humor, Aaronson dicusses various comptational models ranging from the merely strange (Soap bubbles and quantum computers) to the completely wacky (Anthropic computing, where you kill yourself if you don’t get the right answer). He makes a case that the hardness of NP problems should be considered as a physical principle, with interesting predicive consequences.

At 10:56AM on Friday 28th July, our son Hugo Benjamin Miller was born after a long but uncomplicated labor. I was lucky enough to be able to spend the first few nights at the hospital with my wife and Hugo. This was a great pleasure and I think it made things a bit easier for Simone as she recovered from the birth, as she did not have to get in and out of bed to change nappies or settle baby.

Hugo is a wonderful baby (all 4.16kg and 53cm of him) and has so far displayed a very easygoing temperament. He is easy to keep satisfied too, and seems to have only four reasons to cry: nappy, burp, cuddle or food. If I can’t figure out which of these is the cause then I can do a brute-force run through the first three and and then hand him over to Simone, confident that the solution is the one that I can’t deliver. May it stay this way for a long time!

Some pictures of the little man:

We are expecting a baby any hour now (literally), so at last I have an excuse for not updating this thing. I have been a little (sarcasm) busy preparing and moving out of our home ahead of a renovation. Moving out required that I relocate my ADSL connection, and I also took the opportunity to move all of the infrastructure services (email lists, OpenSSH bug tracker, rsync and CVS servers, etc.) from my home server onto a co-located server at Hostcentral. So far this has been very reliable, but I worry about backups now that I no longer have regular physical access to my system.

The scant free time that I have had has been spent trying to figure out how to build CPython modules (notably py-radix, py-editdist and the log reader module in flowd) on Windows XP (gag) with Microsoft’s gratis Visual Studio Express compilers to no avail. I have also been trying to navigate the maze of Python Web Application Frameworks (Django, Pylons, TurboGears, etc., ad nauseum). This too has been a failure, largely because of the “improved” Python setuptools packaging format that all these projects seem to have adopted crashing into my bloody-minded need to build OpenBSD ports/packages of software I install on my systems. Python setuptools distribute modules as “eggs” ostensibly to make things easier for the user (à la CPAN), but they make life quite a bit more difficult for packagers. If I had a weekend to bash at it I could probably knock it over, but that is fantasy. Why am I looking at Python WAFs? I’d like to keep my skills sharp by developing a good cooking recipe site for my wife and some of our friends.

I have updated my OpenBSD TODO list. A few things have been done by others while I have been slacking. Darren Tucker has been busy improving OpenSSH over the last month or so (well, more busy than ever). He has implemented a simple but powerful policy system for sshd_config (search for the Match option). With this, it is possible to do things like:

# Don't trust this guy - only let him use sftp
Match user djm
	AllowTcpForwarding no
	X11Forwarding no
	ForceCommand /usr/libexec/sftp-server -l INFO

What has been implemented so far it pretty basic, but is already useful. It will be better once matching on CIDR address ranges and control of pre-authentication options (in particular authentication types) is added.

I just found, and greatly enjoyed Charles Stross’ A Colder War (full text online) - a very fun bit of science fiction, where the singularity meets the Dark Alliance and H. P. Lovecraft.

The situation in Lebanon saddens me greatly. Hezbollah’s indiscriminate firing of rockets at civilians is wrong and obviously counterproductive. Gandhi showed them the weapons that they should be using against powerful opponents, and demonstrated that they work. On the other hand, Israel’s response is immoral and even more ill-considered. Targeting civilian infrastructure (power stations, ports and airports) is pure state terrorism and a brutal demonstration of military force against a defenceless target. This sort of collective punishment of a nation that was taking tentative steps towards becoming an effective democracy and peacefully disarming Hezbollah will just embitter a generation and guarantee an ongoing supply of recruits to Israel-hating terrorist organisations. If you think that I am being unduly harsh on Israel, consider that Hezbollah is a terrorist organisation and so they cannot be expected to behave with any decency, *unlike* a democracy with a functioning constitution with working courts, UN membership, etc. One cannot justify brutality by saying “my brutal enemy struck first”.

I just wrote and released py-editdist, a fast CPython module to calculate the Levenshtein edit distance between two strings. Wikipedia has a good description of the actual algorithm, and quite a few sample implementations. I looked at using the Python one from there, but it was too slow - taking around ten milliseconds to calculate the distance between two ~50 character strings, which is a problem because I need to do it about 10 million times per day :) py-editdist does the same calculation in ~70 microseconds, which is in the realm of usable at least…

My weekend consisted of cleaning, and recovering from same. Recovering? Well, I used a fair quantity of a product called “Exit Mould” to clean the bathroom and it tore the crap out of my throat and lungs, despite wearing a (light) mask, ventilating the area and limiting the duration of my exposure while the spray was lingering in the air. Today, I have finally stopped coughing up blood but am still a little hoarse.

Apparently, I’m not the only one to have had trouble with this product - Choice magazine reported that a substantial proportion of their testers suffered throat or nose irritation when using it. This shouldn’t be surprising, as the active ingredients are bleach and caustic soda :)

Never again.

After releasing py-bcrypt, I noticed that there is no equivalent for Java. Worse, the top two Google recommendations for “Java password hashing” and “Java password encryption” are really bad: one example uses reversible encryption (using single-DES ECB mode no less), where reversible encryption should only be used when it is really needed and as a last resort. The others recommend the use of an unsalted hash, which allows reverse lookup of the original password from a pre-computed dictionary of digests (a.k.a a “rainbow table”). (I refuse to link to the articles, because by doing so I would further cement their status at the top of Google’s rankings and thereby perpetuate their bad advice)

Why should developers care about the reversability of password encryption or hashing? Because users have been shown to frequently use the same password in multiple places, and if your password database is disclosed then the attacker can revese it at their leisure and then use it to attack other services that your customers use.

I think the best remedy for this situation is to release an implementation of a good password hashing scheme for Java: jBCrypt. This uses the same algorithm as py-bcrypt: Provos and Mazieres’ “A Future-Adapable Password Scheme”, first introduced by OpenBSD and now available in other operating systems (including Solaris 10+). jBCrypt has a really simple API, a license that is very permissive, and a set of JUnit tests to make sure it works.

I just released py-bcrypt, a Python implementation of OpenBSD’s Blowfish password hashing after needing a good password hash for a web application. Hopefully making it available it will encourage people to use it instead of perpetuating stupidities like storing their web application’s passwords unhashed, or as a simple md5(plaintext).