Joe’s Amazing Technicolor Weblog

As promised, I spent yesterday at Organizer's Collaborative's Grassroots Use of Technology conference, up in Lowell. I went hoping particularly to pick up tips on donor management and fundraisining tools, and came away with some good leads. It was also fun to reconnect with folks.

Keynote speakers

Nick Jehlen of Action Mill shared his approach to social change projects, and how that approach played out for Turn Your Back on Bush, Winter Soldier, and Enough Fear. His basic premise is to take Ghandi's idea of being the change that you want to see in the world, and bring it to the commons, so that principled actions have a chance to influence others. In addition to having interesting stuff to say, Nick really put together a handsome presentation, so if you get a chance to catch him speak sometime, go for it.

After lunch, Paul Niwa talked about his Boston Chinatown site, which provides a visualization of the community members' connectedness. He's a professor of journalism at Emerson, so his initial goals were mostly based in journalistic concerns, but one of the interesting results of the project is that it may have provided incentive for some people to become more involved in their community, to boost their importance on the visualiztion! It was also interesting how what Paul called his "journalistic arrogance" led him to publish people's information on the web much more freely than many of us in the nonprofit / activist space would be likely to do.

Sesssions

The first breakout session I went to was horrible. No names, to protect the guilty.

Sura Hart and Katie Winterbottom of Grassroots.org ran a helpful session on SEO. Props to them on running the presentation from Google Apps, on a KDE laptop. As for the content,

• it was helpful to see specifics about keyword research, and the tradeoffs between keyword popularity in searches and the existing presence for that term on the web
• will have to think harder about the working of intrasite hrefs
• hadn't really thought about using the title attribute on tables, forms, etc.
• Google Grants sounds like an amazing opportunity.

Nate Aune of jazkarta had a ton of useful tool suggestions. He started with the constituent database, as that's at the core of almost any successful organization. His recommendation is salesforce.com now that they're giving their service away free to nonprofits. This is significant because it's a best-of-breed solution, with a thriving ecosystem of parters enabled by its comprehensive API.

From there, Nate went on a whirwind tour of helpful tools. I'll only note here the ones that I can see looking into in the future:

Online donations

• we already use paypal, but I'm not sure if we're taking advantage of the fact that they give nonprofits a lower fee than for-profits. Also, all processing can be done via their API - no need to send folks to paypal.com's ugly pages (which we do now...)
• fundable
• chipin
• for nonprofits, google checkout is totally free through next year. Interesting, I wonder what happens then?

Mass email

• campaign monitor - we've just been giving them a try, so it was nice to hear that jazkarta has good luck with them. However:
• VerticalResponse is also supposed to be excellent, and is integrated to salesforce.com

Misc.

• eventbrite sounds very handy for online ticketing
• phone.com's integration of voice mail and email could be very handy

In the market for a reliable, cheap, low power firewall, and already familiar with OpenBSD & PF, I've been eying the Soekris line of products. This project looks like it'll be just a little more involved than buying something off the shelf, but way more flexible, and have more parts in common with other stuff I'm already using.

From my research, it looks like the necessary pieces are:

• Soekris 4501 & case - $173
• power supply - $11
• 2 GB SanDisk CF - $26
• USB card reader - $12 (to write to the CF card, naturally)
• Null Modem & USB <-> DB9 cable - $27 (no hardware with DB9 around)

So, for about $250 + some shipping, and a bit of fiddling around time, this could be a pretty robust solution. I'm not sure I'd care to work out all the installation and configuration details myself, but there are a couple guides to getting everything up & running. I'll certainly add my own notes if I go through with this.

I spent part of last Wednesday at the Boston Tech-Security Conference, held at lovely UMass Boston. Getting me there was a triumph of email marketing by hosts Data Connectors, as I hadn't heard of this series and couldn't really find any 3rd party accounts on the web. That, and it was free. Since I'm newly back in a higher-level IT administrative capacity after years of just focusing on web application security, I decided to give the conference a try.

First impressions (after noting how unfriendly the UMass campus is to bicycles) were of an absence of buzz. Many vendors hanging around in one room, and then a vendor representative giving a traditional one-way presentation in the next room. After attending a few barcamps, it really seems to me that a more interactive format would benefit everyone involved. This conference was clearly driven by the vendors, but it seems to me that it'd be in their best interests to learn more about their potential customers'  interests and concerns, rather than broadcasting a sales message.

Despite the one-to-manyness, the presentations I made it to varied quite a bit. A few did a good survey of some aspect of security, and then tied that discussion into the vendor's offerings right at the end. Others were basically just sales pitches. Guess which kind lost more of the audience?

new concepts to me

NAC - Network Admission Control. Folks will sell you systems that go beyond requiring just a username / password combo to get on a given network, by combining checks on device MAC / IP, allowed hours of operation, and presence and activity of specified software. Packetfence looks like a promising open source NAC.

IPS - Intrusion Prevention System. If I'm understanding correctly, these go beyond IDSes by taking some action such as updating firewall rules when naughtiness is detected. It looks like Snort has been able to do this sort of thing for a while, also PF has some related capabilities.

notable speakers

Ming Fu of Lumension introduced me to the concept of thumbsucking -- apparently the new hotness in social engineering attacks is to leave USB drives with said software lying about in parking lots, expecting that some percentage will be picked up & plugged in... nasty! Lumension's tie-in is that they have a product that allows strictly defined device access controls for windows boxes, so you could set up rules that would prevent employee accounts from mounting any USB devices, and only allow admins to mount USB devices already encrypted with your organization's key.

Ken Pappas of Top Layer Networks gave a high-level rundown on the overall tech security situation, and managed to do it with not an ounce of sales pitch. Authentic confidence is an excellent marketing tool, and Ken's got that. His early remarks included a shout-out to the Boston chapter of the possibly shooting-to-kill InfraGard. He then went on to summarize the state of security in '08, which is basically: not that great. Incidence rates are going up, the range of attackers is increasing in professionalism and skill at the top end and becoming even less sophisticated (i.e. lower barrier to entry) on the low-end with easily available point-and-click tools for launching mail bombs, etc. Those of us responsible for computers attached to networks definitely need to be budgeting some of our time to keep up with the evolving threats, regardless of whether we're aware of any particular adversary that's out to get us or our data.

takeaway

It's important to get away from the daily stream of projects periodically, to think about things from a higher level. Vendory as this conference was, it did give me that opportunity, and I did re-prioritize my TODO list at the end of the day.

The downside: I hadn't thought through how many calls & emails I was setting myself up for receiving from the conference vendors. FYI, I am 100% of the time never going to spend money with someone who calls me out of the blue and interrupts whatever I'm working on. Send me an email, and I'll file it for processing at an appropriate time.

On my agenda this weekend is the '08 edition of the Grassroots Use of Technology conference, happening up in Lowell. I was a volunteer at the conference back in '04 and '05, but I've been out of town for the last couple.

This year I'll be wearing my IT Manager hat & looking to pick people's brains particularly about mass emailing, online donations, and fundraisining tools.

Some context -- my new gig features a big 'ol FileMaker installation, which has a number of automated maintenance routines. Some of those routines send emails, through a convoluted process involving FileMaker calling a GUI MUA (e.g. Mail.app). There are a number of practical problems with this, such as the requirement of another computer running, always logged into an account capable of sending the emails.

We'd done a bit of research about strictly server-side alternatives, and found surprisingly little. The best resource was Graham Sprague's page about sending emails via FileMaker's XSLT Web Publishing tool. We gave that approach a try, but didn't get any results, or anything useful from FileMaker's logs to explain why things weren't working. I'm not sure what FileMaker version Graham's example was written for, perhaps something's changed with version 9?

Rather than dive into FileMaker's proprietary XSLT system to debug things, it occurred to me that this might be a job for FileMaker's PHP API. Sure enough, after about 15 minutes of consulting the API Doc, we were sending emails based on the contents of a FileMaker record.

We're still working on ironing out the details, but here's the rough proof of concept PHP file. It works with the example email database from Graham's XSLT sample, with the php permission added to the database. Plenty of missing features such as cc & bcc fields, actually checking for the 'send' flag, checking for errors, any kind of authorization or authentication, etc. In other words, you probably don't want to be running this on a publicly accessible webserver, but at least it presents the basic idea in a simple form.

<?php
require_once('FileMaker.php');
$fm = new FileMaker();
$fm->setProperty('username', 'send_email');
$fm->setProperty('password', 'whatever_the_password_is');
$fm->setProperty('database', 'Email');

$findCmd =& $fm->newFindAllCommand('Utility_Email');
$result = $findCmd->execute();
$records = $result->getRecords();
foreach($records as $record) {
  $headers = array("From: " . $record->getField('from'));

  mail($record->getField('to'),
       $record->getField('subject'),
       $record->getField('message'),
       implode("\r\n", $headers)
      );
}
echo "Emails sent";

As regular readers saw earlier, I've been casting about trying to find an open source backup solution that handles OS X metadata reliably. Having been disappointed by rdiff-backup, I've turned by eyes to the similar rsnapshot project, which uses the venerable rsync.

Since macports includes the latest version of rsync, 3.0.2, I gave it a try with the familiar rsync -avz /from /to syntax, but it performed disappointingly on n8's handy Backup Bouncer test suite. Thanks to Mike Bombich, I learned about some extra flags to add (though my copy of rsync doesn't seem to know about the -N or --fileflags he has):

$ sudo rsync -aHAXx  /Volumes/Src/ /Volumes/rsynctest/
$ ./bbouncer verify -d /Volumes/Src/ /Volumes/rsynctest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... ok
ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok
ok

Sure, there are a few FAILS in there, but they're not important:

$ ./bbouncer verify -T important -d /Volumes/Src/ /Volumes/rsynctest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... ok
ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok
   Sub-test:              on dirs ... ok
ok

(note the -T important flag telling Backup Bouncer to remove the extra-finicky tests). Good enough! On to get familiar with rsnapshot.

I've finally had the chance to repeat my test of the rdiff-backup-devel package from MacPorts, using the same steps as I used for the stable package. The results are better but not fantastic:

$ ./bbouncer  verify -d /Volumes/Src/ /Volumes/rdifftest
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... FAIL
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok
   Sub-test:       on directories ... ok
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
FAIL
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... FAIL
FAIL

With promising reports out on rsync 3, looks like it's time to take another look at rsnapshot...

Preamble: after a bunch of research, rdiff-backup looked like a good open source solution for OS X backup. And it's in macports!

Wrinkle: after getting further into my current backup project, I've become more aware of the difficulty of getting all of the various forms of OS X metadata backed up. Some research revealed the tool Backup Bouncer, which provides a way to run some tricky tests on any OS X backup system. Here I'm using the latest bbouncer (0.1.2), on a 10.4.11 system all patched up through security update 2008-03.

The stable version of rdiff-backup in macports is 1.0.5, which is what's being used for the following tests. I've poked at both this and the rdiff-backup-devel version, which is 1.1.15, but it doesn't automatically pull in the xattr module from macports. I'll retest with that version later. Also note that the bbouncer results don't suggest that xattr is doing 1.0.5 much good.

$ ./bbouncer create-vol rdifftest
$ sudo rdiff-backup /Volumes/Src /tmp/bb
$ sudo rdiff-backup --force -r 1D /tmp/bb /Volumes/rdifftest/
$ ./bbouncer verify -d /Volumes/Src/ /Volumes/rdifftest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok
Verifying:       resource-forks ... FAIL
Verifying:         finder-flags ... FAIL
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL
   Sub-test:       on directories ... FAIL
   Sub-test:          on symlinks ... FAIL
FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL
   Sub-test:              on dirs ... FAIL
FAIL
Verifying:                 fifo ... FAIL
Verifying:              devices ... FAIL
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL
FAIL

That's a whole lot of FAIL, especially compared to the built-in OS X rsync, which passes on resource forks and finder flags, extended attributes for files and directories, ACLs, and the last few tests.

Granted, there are a bunch of things being tested by bbouncer that may or may not be of interest to the average user. A recent post by the author points out that bbouncer has a -T flag for indicating which level of paranoia to run at. Rerunning the test for rdiff-backup 1.0.5 with only critical tests is somewhat instructive:

$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/rdifftest/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:       resource-forks ... FAIL
Verifying:         finder-flags ... FAIL

Compare to OS X's rsync:

$ ./bbouncer verify -d -T critical /Volumes/Src/ /Volumes/Dst/10-rsync-apple/
Verifying:    basic-permissions ... ok
Verifying:           timestamps ...
   Sub-test:    modification time ... ok
ok
Verifying:             symlinks ... ok
Verifying:       resource-forks ... ok
Verifying:         finder-flags ... ok

Unfortunately my camera phone is too lame to pick out the small print on the back of the box, but it includes -- wait for it -- "Mints made in Canada".

Oh, the disappointment.

Thanks to a post on Slice, shared at an opportune time by renowned pizza guru Max, I was inspired to fire up the grill this weekend & cook some pizza. Thanks to gribley for documenting & daring to eat said experiment.

As you might imagine, it was an exciting process. Getting the heat just right in our tiny scavenged grill is pretty tough, so there was a bit more char than I would've liked. Flipping that dogg was a trick, too. However, the finished product was surprisingly edible. In toto, an experiment worth repeating.