• I’d forgotten how to figure out the Soekris’ MAC address, which is needed for the DHCP server config. Turns out it’s shown when you tell the soekris to try netbooting via boot f0.
• The ubuntu (& presumably debian) tftpd configures itself without the -s flag, which allows pxeboot’s requests for files like /bsd to be found in the /srv/tftp/ directory
• Despite rediscovering the above, I ended up reinstalling onsite from my macbook. Got most of the way thanks to tfpd tips here and the ISC dhcp server from MacPorts. However, as launchd was involved, there was much unhappiness getting tftpd to actually serve the files in question. Had just about thrown up my hands when I figured out that the -s flag actually works the same as it does in debian, not the way the OS X manpage says it does.
• After close to 10 years of being confused about why OpenBSD’s installer often pukes when trying to use a local ftp or http connection, I finally thought to look at my local webserver’s access log, and saw that the installer was trying to get an index.txt file. Creating one with the name of the relevant files did the trick.
• I thought to plug the Soekris into a Kill a Watt while installing, and never saw a draw of over 4 watts while formatting the CF, copying over the install set, or writing the files. Not bad! Idles at 2 watts.

Rather than taking Michiel’s exact approach with the memory filesystem, I decided I’d use the memoryfs for the commonly-changed files, but leave the root writeable, which has the benefit of allowing ports to be installed and configuration changes to be made on the fly.

1. Which soekris? Current needs are reasonably simple firewalling for a smaller office, so the 4501 answers nicely. Would consider 5501s for an office with more traffic / servers.
2. What media? 2 GB CF cards are cheap as dirt now, I remember running OpenBSD on systems with smaller hard drives… the lower heat / power / footprint and higher reliability of CF is very attractive for this application, compared to a hard drive.
3. What OS / distro? Initially, I’m going with what’s basically a default i386 OpenBSD installation. Commonly used for firewalls, and given the cheapness and reliability of CF these days, a generic setup should work for now. Once I’m comfortable with how things are running I’ll look into more options for minimizing writes, and possibly mounting some or all of the FSes ro. The most promising reference I’ve found so far is Michiel van Baak’s guide.
4. What style of installation? Having serial access seems like a much better idea than just blindly booting CF cards, for troubleshooting purposes if nothing else. This also flows from the previous question, if I’d settled on a flashdist type approach, I’d be writing via a CF card. As it was, I got a keyspan USB-to-serial connector (drivers for OS X and linux!) & a null modem, and talked to the 4501 via screen /dev/cu.KeySerial1 19200 from my laptop. I then booted the Soekris from a convenient Debian server, with some helpful tips from here.

Stumbling blocks I ran into:

• The whole TFTP booting situation was tricky due to a missing ‘next-server’ directive in my dhcpd conf file, which I fixed thanks to this.
• After that, I got hung at

  
  boot> boot bsd.rd
  booting tftp:bsd.rd: 4780308+874136 [52+178240+163973]=0x5b821c
  entry point at 0x200120

  which was fixed by doing

  
  boot> set tty com0
  boot> stty com0 19200

  before booting bsd.rd (thanks to google’s cache of an mailing list discussion for that).

From my research, it looks like the necessary pieces are:

• Soekris 4501 & case - $173
• power supply - $11
• 2 GB SanDisk CF - $26
• USB card reader - $12 (to write to the CF card, naturally)
• Null Modem & USB <-> DB9 cable - $27 (no hardware with DB9 around)

So, for about $250 + some shipping, and a bit of fiddling around time, this could be a pretty robust solution. I’m not sure I’d care to work out all the installation and configuration details myself, but there are a couple guides to getting everything up & running. I’ll certainly add my own notes if I go through with this.