Joe’s Amazing Technicolor Weblog

I’ve been trying out Mac OS 10.6 a.k.a. Snow Leopard for a few weeks now. For the most part it looks and acts… just like Leopard! Still, I have run into the following annoyances:

• Doesn’t really want to do more than one thing if you only have 1 GB RAM, very noticeably worse than Tiger in this regard (never ran Leopard much on only 1 GB).  I guess there are more ints running in the OS & in basic apps than I would have thought, if it is the 64bitness to blame.
• Doesn’t work with our older b/g Airport Extreme. Says it’s on the wireless network, but doesn’t configure TCP/IP settings — this is after much experimenting with various Airport settings. Search for ‘snow leopard wireless’ for a variety of related complaints.
• Doesn’t work with the Citrix XenApp web plugin. To be fair, this seems to be due to Citrix expecting Java 1.5 to be installed, which is kind of lame. Workarounds are reported on the internets, but then you’re managing your own Java installation, which seems to be one of the most vulnerability-plagued pieces of OS X.

My conclusion, as of 10.6.2: no reason to upgrade from Leopard, unless you’ve bought brand-new hardware that requires SL.

Here’s the configuration I’ve been working on: control access to Apache webserver by checking (over LDAP) against our existing user database, held in an OS X Open Directory. It’s taken me more casting about than I’d expected, but it looks like I’m finally there.

In the beginning, I got a little confused by the HTTP auth options. I’d been hoping to use Digest mode, but a comment on this post points out the logical problem with that: Digest doesn’t involve the password making its way to Apache, so there’s no way for it to pass the password along over LDAP.

BTW this is under Tiger (OS X 10.4) — I’m not sure if anything changes with other versions of OS X.

Once figuring out that I did need to use Basic auth, Production Monkeys got me most of the way with my Apache config. What I missed is that, at least with our OD configuration, it’s necessary to include the server name in the dc list. Here’s what worked for me:

<Location "/somewhere">
AuthType Basic
AuthName "Whatever You Call This Auth"
Require valid-user
AuthBasicProvider ldap
AuthLDAPURL ldap://servername.yourdomain.org/cn=users,dc=servername,dc=yourdomain,dc=org?uid
AuthzLDAPAuthoritative off
 </Location>

Because rsync 3 is one of if not the only OS X backup solutions that actually gets all the possible forms of metadata, I’m in the market for a backup solution that uses it. The current candidate is rsnapshot, a 6000 line perl program (!), conveniently located in macports, that wraps rsync to do smart backup things like keep snapshots via hard links. Thanks to the O’Reilly Backup book for pointing me to it.

• It’s pretty easy to set up to run locally. The main trick is that the configuration file requires tabs. First time in ~10 years I’ve had to turn tabs back on in vim (:set noexpandtab).
• For OS X, we want the magic -aNHAXx --fileflags --force-change args to make rsync behave properly with all the metadata.
• OS X has a weird directory structure, so if you try to backup /etc you just get the symlink that is to /private/etc, or if you try to exclude something under /var, you miss it because it’s really /private/var/bigdirectory. Buzzkill.
• Lchown.pm is necessary for symlinks to have the right ownership in snapshots. CPAN’d.
• Running things automatically on OS X as the privileged user is a bit odd, lacking as it does a traditional root account. Sudo does nicely, with NOPASSWD: /opt/local/bin/rsnapshot
on the backup machine. For the clients we need appropriate ssh settings, with some tricks to run sudo on the remote machine. Getting this running took a while, since I missed the fact that running rsnapshot via sudo on the backup machine meant that rsync would try to use root’s ssh key, not the backup user’s — fixed this with the -i arg to ssh.
• Restoring backups is just a matter of copying them from the appropriate snapshot dir, probably using rsync & the same arguments rsnapshot uses (easily extracted from the rsnapshot log).
• Multiple servers are done serially. If you wanted to run backups in parallel, you’d need one configuration file for every server you’re backing up, and they each need their own snapshot_root, logfile, and lockfile. For my installation, this’d be more trouble than it’s worth.
  

Another issue that temporarily gave me pause, now that we’ve got everything backed up with presumably correct permissions, is that our off-site backup procedures involve creating tar files & encrypting them. Fortunately, backup bouncer shows that OS X tar gets all the important stuff right, though it’s no rsync v3.

Update: I’d forgotten to check the flags that had been missing from the macport version last time around, but they’re all there. Now, running rsync -aNHAXx --fileflags --force-change /Volumes/Src/ /Volumes/rsync3test passes every bbouncer test! Cool.

Original post: On Leopard (10.5.5) using rsync 3.0.4 from macports, a few more tests are passing backup bouncer 0.1.3 (compare to my June post):

$ sudo ./bbouncer verify -d /Volumes/Src/ /Volumes/rsync3test/
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... ok (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok

As regular readers saw earlier, I’ve been casting about trying to find an open source backup solution that handles OS X metadata reliably. Having been disappointed by rdiff-backup, I’ve turned by eyes to the similar rsnapshot project, which uses the venerable rsync.

Since macports includes the latest version of rsync, 3.0.2, I gave it a try with the familiar rsync -avz /from /to syntax, but it performed disappointingly on n8’s handy Backup Bouncer test suite. Thanks to Mike Bombich, I learned about some extra flags to add (though my copy of rsync doesn’t seem to know about the -N or — fileflags he has):

$ sudo rsync -aHAXx  /Volumes/Src/ /Volumes/rsynctest/
$ ./bbouncer verify -d /Volumes/Src/ /Volumes/rsynctest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... ok
ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok
ok

Sure, there are a few FAILS in there, but they’re not important:

$ ./bbouncer verify -T important -d /Volumes/Src/ /Volumes/rsynctest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... ok
ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok

(note the -T important flag telling Backup Bouncer to remove the extra-finicky tests). Good enough! On to get familiar with rsnapshot.

I’ve finally had the chance to repeat my test of the rdiff-backup-devel package from MacPorts, using the same steps as I used for the stable package. The results are better but not fantastic:

$ ./bbouncer  verify -d /Volumes/Src/ /Volumes/rdifftest
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... FAIL
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
FAIL
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... FAIL
FAIL

With promising reports out on rsync 3, looks like it’s time to take another look at rsnapshot…

Preamble: after a bunch of research, rdiff-backup looked like a good open source solution for OS X backup. And it’s in macports!

Wrinkle: after getting further into my current backup project, I’ve become more aware of the difficulty of getting all of the various forms of OS X metadata backed up. Some research revealed the tool Backup Bouncer, which provides a way to run some tricky tests on any OS X backup system. Here I’m using the latest bbouncer (0.1.2), on a 10.4.11 system all patched up through security update 2008-03.

The stable version of rdiff-backup in macports is 1.0.5, which is what’s being used for the following tests. I’ve poked at both this and the rdiff-backup-devel version, which is 1.1.15, but it doesn’t automatically pull in the xattr module from macports. I’ll retest with that version later. Also note that the bbouncer results don’t suggest that xattr is doing 1.0.5 much good.

$ ./bbouncer create-vol rdifftest
$ sudo rdiff-backup /Volumes/Src /tmp/bb
$ sudo rdiff-backup --force -r 1D /tmp/bb /Volumes/rdifftest/
$ ./bbouncer verify -d /Volumes/Src/ /Volumes/rdifftest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... FAIL
Verifying:         finder-flags ... FAIL
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL
   Sub-test:       on directories ... FAIL
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
FAIL
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL
FAIL

That’s a whole lot of FAIL, especially compared to the built-in OS X rsync, which passes on resource forks and finder flags, extended attributes for files and directories, ACLs, and the last few tests.

Granted, there are a bunch of things being tested by bbouncer that may or may not be of interest to the average user. A recent post by the author points out that bbouncer has a -T flag for indicating which level of paranoia to run at. Rerunning the test for rdiff-backup 1.0.5 with only critical tests is somewhat instructive:

$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/rdifftest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:       resource-forks ... FAIL
Verifying:         finder-flags ... FAIL

Compare to OS X’s rsync:

$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/Dst/10-rsync-apple/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok

Here’s my top 10 command line entries, according to history | awk '{a[$2]++} END{for(i in a){printf "%5d\t%s\n",a[i],i}}' | sort -rn | head

125 svn
58 ls
38 symfony
37 gvim
36 vim
36 cd
34 php
20 uptime
14 ssh
10 lighttpd

Idea & shell magic via Bill de Hóra. You can see that my recent work has been symfony-heavy (that’s what all the php is for, too).