Joe’s Amazing Technicolor Weblog

Having a few accounts at slicehost has sold me on the potential of server virtualization. Faced with the need to build a new in-house server on some older hardware, I figured it’d be worth taking a look at setting the new server up as a virtual host. Even though I only need one linux server today, it’d be nice to easily migrate  sets of services on/off it in the future, jump onto new hardware without doing a whole reinstall, etc.

I like ubuntu, so that’s where I started looking at for the host OS. Being on the client end of Xen via slicehost has been smooth, but there are also other virtualization options pushed in the Ubuntu docs, including at least VMware and KVM, so I’ve spent a little time looking into them. I’m a bit of a Free Software snob, so VMware was off the list. KVM requires one of a small set of recent processors to run — the CPU of the server in question is on that list, so KVM remained an option.

Due to previous experience, though, I started looking at Xen first, only to find that Ubuntu isn’t and won’t be supporting Xen from the current release onwards. Wondering why that would be, I came to the following debate, and became thoroughly confused.

After getting this far into the process, I revisited my reasons for looking into virtualization in the first place, and came to the conclusion that my relatively simple needs don’t justify the time to wade through the options.

In a word, no.

However, old-skool readers¹, you can still rejoice: the slagwerks blog homepage has a new feature that will help you answer this question any time you like. In honor of Earth Day 2009, you can now see at the above URL a shiny JPG depicting the latest CO₂ levels recorded by NOAA’s lab at the Mauna Loa Observatory in Hawaii.

To celebrate, here’s the mega version.

---

1: old-skool — reading this via a web browser rather than some sort of feed reader

Faced with a big site full of URLs like http://mysite.com/Internal1.asp?id=357 to mirror & archive, I recently tried out a new (to me) tool, HTTrack. I’ve fiddled with wget for this sort of job in the past, but it always takes me ages of man-page reading to get my options right, and even then not everything seems to work out.

This time around, for example, I’d convinced myself that wget -r -N -l inf --no-remove-listing -E -k -p http://mysite.com would do the trick. It mostly did, except for seemingly random pages that didn’t get all of their links converted.

HTTrack, on the other hand, did The Right Thing without any switches or arguments whatsoever. It was a bit more of a pain to get running; even though it’s in macports, right now the port is lagging behind the available versions, so I had to actually type ./configure and ./make myself. Well worth it for a usable mirror.

The Soekris I installed a few months ago has been working well, so I’m setting up a second for another location. Mostly setting it up the same as before, and collected the following notes along the way.

• I’d forgotten how to figure out the Soekris’ MAC address, which is needed for the DHCP server config. Turns out it’s shown when you tell the soekris to try netbooting via boot f0.
• The ubuntu (& presumably debian) tftpd configures itself without the -s flag, which allows pxeboot’s requests for files like /bsd to be found in the /srv/tftp/ directory
• Despite rediscovering the above, I ended up reinstalling onsite from my macbook. Got most of the way thanks to tfpd tips here and the ISC dhcp server from MacPorts. However, as launchd was involved, there was much unhappiness getting tftpd to actually serve the files in question. Had just about thrown up my hands when I figured out that the -s flag actually works the same as it does in debian, not the way the OS X manpage says it does.
• After close to 10 years of being confused about why OpenBSD’s installer often pukes when trying to use a local ftp or http connection, I finally thought to look at my local webserver’s access log, and saw that the installer was trying to get an index.txt file. Creating one with the name of the relevant files did the trick.
• I thought to plug the Soekris into a Kill a Watt while installing, and never saw a draw of over 4 watts while formatting the CF, copying over the install set, or writing the files. Not bad! Idles at 2 watts.

Rather than taking Michiel’s exact approach with the memory filesystem, I decided I’d use the memoryfs for the commonly-changed files, but leave the root writeable, which has the benefit of allowing ports to be installed and configuration changes to be made on the fly.

Am I crazy, or does Michael Franti’s song Hello Bonjour have a reaggeton beat? The drum sounds are a bit different than we’re used to from hanging out in Hyde Square, but I think that’s still it. Yes? No?

In short, I’m happy to report that passenger a.k.a. mod_rails is awesome. I see no need to fiddle with mongrel, mongrel_cluster & mod_proxy for future rails apps.

Miscellaneous notes from the project:

• My environment is now ubuntu server 8.04, on slicehost. This project finally prompted me to upgrade from 7.10, which went very smootly — the long-term support releases of Ubuntu seem like the way to go for most server situations.
• How to handle the apache config? We have some php stuff running on this server, too, so the 3rd party ubuntu passenger package, which wants to use the worker mpm, isn’t the hot ticket. No problem, passenger’s default gem-based install is smooth as butter.
  This configuration also seems to require RailsAutoDetect off, which had the side benefit of letting me leave an existing mongrel-using configuration alone until I had a few minutes hours to upgrade it to rails 2.2.2, then switch it over to git & passenger.
• Slicehost has decent docs for this, in particular ubuntu-hardy-mod_rails-installation and ubuntu-hardy-using-mod_rails-to-serve-your-application & its comments.
• Capistrano mods for passenger
• This was also my first git-powered rails app, which proved a little challenging to get going with cap, but has turned out really nicely. Very fast deployments.
  Along the way, having a plugin as a git submodule proved to be more trouble than it was with, though I’m not sure if that’d apply to my final configuration, which involves checking the project out from the same server as it’s deployed on,  using the :local_repository argument to capistrano.
• This project helped me realize that having a deployed app as a public github project is more trouble than it’s worth, what with the various configuration informations that I’d rather have under SCM but don’t want to share.

I’ve used technoweenie’s handy us_state_select rails plugin in past projects. Just tried it out on a new rails 2.2 project & found out that the call to InstanceTag needed to be updated, and since technoweenie’s SVN version of the plugin seems moribund, I thought I’d take 30 extra seconds to put it up on github. Fork away!

P.S. github is way cool.

or, Why To Use Dedicated Layouts When Connecting To FileMaker Via PHP

I’d read that it’s a good practice to always use a dedicated layout for any PHP scripts you have that are talking to a FileMaker database. While I’d seen reasons of efficiency and reliability, today I learned another reason that’s true: it can eliminate otherwise hard-to-debug problems.

At first when working on my current FileMaker <-> PHP project, I was attempting to reuse an existing layout that had all the info I needed. While my permissions seemed to be fine for the data file and layout I was attempting to access, actually running the script kept resulting in “Error 100: File is missing” coming back at me as soon as I added any criteria to my search. FileMaker doesn’t bother putting anything useful in its server logs, either, so it wouldn’t have been much fun picking through the layout & figuring what linkage(s) were to blame.

However, by simply creating a dedicated layout, everything started working as planned. A practice I’ll be following in the future.

I’ve had a number of experiences setting up rich text editors online over the last few years, usually in the context of a Drupal site, and it’s always been a pain.

My most recent experience has mostly been the same, except it ends a little better than usual:

1. Start with TinyMCE, which has more-or-less worked in the past. This time around with Drupal 6, unfortunately, my experience was mostly less: it just wouldn’t respond to the customization settings I was setting in the Drupal backend.
2. Do some more research, find excitement about the YUI editor. It seemed promising, but ultimately failed to allow image uploads despite hours of fiddling.
3. Turn to the other oft-mentioned option, FCKeditor. After a few minutes of fiddling supported by the included readme.txt, it’s actually working and uploading images easily. Amazing!

Not everything in FCK is customizable through the web, but that’s fine. It’s probably easier to comment out a line in fckeditor.config.js, anyhow. I started with the DrupalFull toolbar & took off a few things we don’t want.

Since versions are so often key with these things, I’m using Drupal 6.6, fckeditor-6.x-1.3-rc3 (the drupal module), FCKeditor_2.6.3 (the javascript bit).

Including a reasonably debugged WYSIWYG in the Acquia distribution would get me to take it for a test drive next time around, because this cycle is such an amazing waste of time whenever I go through it.

Because rsync 3 is one of if not the only OS X backup solutions that actually gets all the possible forms of metadata, I’m in the market for a backup solution that uses it. The current candidate is rsnapshot, a 6000 line perl program (!), conveniently located in macports, that wraps rsync to do smart backup things like keep snapshots via hard links. Thanks to the O’Reilly Backup book for pointing me to it.

• It’s pretty easy to set up to run locally. The main trick is that the configuration file requires tabs. First time in ~10 years I’ve had to turn tabs back on in vim (:set noexpandtab).
• For OS X, we want the magic -aNHAXx --fileflags --force-change args to make rsync behave properly with all the metadata.
• OS X has a weird directory structure, so if you try to backup /etc you just get the symlink that is to /private/etc, or if you try to exclude something under /var, you miss it because it’s really /private/var/bigdirectory. Buzzkill.
• Lchown.pm is necessary for symlinks to have the right ownership in snapshots. CPAN’d.
• Running things automatically on OS X as the privileged user is a bit odd, lacking as it does a traditional root account. Sudo does nicely, with NOPASSWD: /opt/local/bin/rsnapshot
on the backup machine. For the clients we need appropriate ssh settings, with some tricks to run sudo on the remote machine. Getting this running took a while, since I missed the fact that running rsnapshot via sudo on the backup machine meant that rsync would try to use root’s ssh key, not the backup user’s — fixed this with the -i arg to ssh.
• Restoring backups is just a matter of copying them from the appropriate snapshot dir, probably using rsync & the same arguments rsnapshot uses (easily extracted from the rsnapshot log).
• Multiple servers are done serially. If you wanted to run backups in parallel, you’d need one configuration file for every server you’re backing up, and they each need their own snapshot_root, logfile, and lockfile. For my installation, this’d be more trouble than it’s worth.
  

Another issue that temporarily gave me pause, now that we’ve got everything backed up with presumably correct permissions, is that our off-site backup procedures involve creating tar files & encrypting them. Fortunately, backup bouncer shows that OS X tar gets all the important stuff right, though it’s no rsync v3.