the stable rdiff-backup is not looking so hot for metadata on OS X
Preamble: after a bunch of research, rdiff-backup looked like a good open source solution for OS X backup. And it’s in macports!
Wrinkle: after getting further into my current backup project, I’ve become more aware of the difficulty of getting all of the various forms of OS X metadata backed up. Some research revealed the tool Backup Bouncer, which provides a way to run some tricky tests on any OS X backup system. Here I’m using the latest bbouncer (0.1.2), on a 10.4.11 system all patched up through security update 2008-03.
The stable version of rdiff-backup in macports is 1.0.5, which is what’s being used for the following tests. I’ve poked at both this and the rdiff-backup-devel version, which is 1.1.15, but it doesn’t automatically pull in the xattr module from macports. I’ll retest with that version later. Also note that the bbouncer results don’t suggest that xattr is doing 1.0.5 much good.
$ ./bbouncer create-vol rdifftest
$ sudo rdiff-backup /Volumes/Src /tmp/bb
$ sudo rdiff-backup --force -r 1D /tmp/bb /Volumes/rdifftest/
$ ./bbouncer verify -d /Volumes/Src/ /Volumes/rdifftest/
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: symlink-ownership ... ok
Verifying: hardlinks ... ok
Verifying: resource-forks ... FAIL
Verifying: finder-flags ... FAIL
Verifying: finder-locks ... FAIL
Verifying: creation-date ... FAIL
Verifying: bsd-flags ... FAIL
Verifying: extended-attrs ...
Sub-test: on files ... FAIL
Sub-test: on directories ... FAIL
Sub-test: on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
Sub-test: on files ... FAIL
Sub-test: on dirs ... FAIL
FAIL
Verifying: fifo ... FAIL
Verifying: devices ... FAIL
Verifying: combo-tests ...
Sub-test: xattrs + rsrc forks ... FAIL
Sub-test: lots of metadata ... FAIL
FAILThat’s a whole lot of FAIL, especially compared to the built-in OS X rsync, which passes on resource forks and finder flags, extended attributes for files and directories, ACLs, and the last few tests.
Granted, there are a bunch of things being tested by bbouncer that may or may not be of interest to the average user. A recent post by the author points out that bbouncer has a -T flag for indicating which level of paranoia to run at. Rerunning the test for rdiff-backup 1.0.5 with only critical tests is somewhat instructive:
$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/rdifftest/
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: resource-forks ... FAIL
Verifying: finder-flags ... FAILCompare to OS X’s rsync:
$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/Dst/10-rsync-apple/
Verifying: basic-permissions ... ok
Verifying: timestamps ...
Sub-test: modification time ... ok
ok
Verifying: symlinks ... ok
Verifying: resource-forks ... ok
Verifying: finder-flags ... ok
June 6th, 2008 at 2:49 pm
Glad to see BB is useful to you. I’m wondering why you did this two-step copy though (Sorry if the formatting doesn’t come through):
$ sudo rdiff-backup /Volumes/Src /tmp/bb
$ sudo rdiff-backup — force -r 1D /tmp/bb /Volumes/rdifftest/
Perhaps you’re trying to simulate a backup/restore cycle? If so, you should probably use another bbouncer create-vol volume for the intermediate step instead of /tmp, just to be confident in the results. This is especially true on Tiger, since ACLs weren’t even enabled on the root filesystem by default IIRC.
If, after doing that, you still get failures with “critical” features like resource forks and finder flags you should talk to the rdiff backup people, since they seem to think it should work: http://www.nongnu.org/rdiff-backup/FAQ.html#OSX
June 6th, 2008 at 4:09 pm
Best I can understand, rdiff-backup backups aren’t intended to be directly used; instead, to get the original file tree you need to run a restore (rdiff-backup -r TIME /backup /destination) from the backup.
In theory, I don’t believe it should matter if the backup filesystem (/tmp/bb in this case) doesn’t have all the same capabilities as the target / destination filesystem, as rdiff-backup has its own store for metadata. Just to make sure, I just tried following your suggestion of backing up to a volume created by bbouncer, but got the same results.
I’m not sure what version of rdiff-backup got the extended OS X goodies, so it may be that the stable version available through macports is just too old; will find out once I’ve got the bandwidth to pull down the new version & retest.
Thanks a bunch for BB! Very handy!
June 9th, 2008 at 5:22 pm
[…] Joe’s Amazing Technicolor Weblog « the stable rdiff-backup is not looking so hot for metadata on OS X […]
September 7th, 2008 at 10:00 am
You might as well be interested in mlbackup, it includes rsync 3 (which passes ALL bbouncer tests). mlbackup makes rsync based disk-to-disk backups convenient with hardlinks to save space and speed up incrementals, autorotation and email notification. mlbackup is released under GPLv3.
Best Regards
Pepi (author of mlbackup)