Joe’s Amazing Technicolor Weblog

or, Why To Use Dedicated Layouts When Connecting To FileMaker Via PHP

I’d read that it’s a good practice to always use a dedicated layout for any PHP scripts you have that are talking to a FileMaker database. While I’d seen reasons of efficiency and reliability, today I learned another reason that’s true: it can eliminate otherwise hard-to-debug problems.

At first when working on my current FileMaker <-> PHP project, I was attempting to reuse an existing layout that had all the info I needed. While my permissions seemed to be fine for the data file and layout I was attempting to access, actually running the script kept resulting in “Error 100: File is missing” coming back at me as soon as I added any criteria to my search. FileMaker doesn’t bother putting anything useful in its server logs, either, so it wouldn’t have been much fun picking through the layout & figuring what linkage(s) were to blame.

However, by simply creating a dedicated layout, everything started working as planned. A practice I’ll be following in the future.

I’ve had a number of experiences setting up rich text editors online over the last few years, usually in the context of a Drupal site, and it’s always been a pain.

My most recent experience has mostly been the same, except it ends a little better than usual:

1. Start with TinyMCE, which has more-or-less worked in the past. This time around with Drupal 6, unfortunately, my experience was mostly less: it just wouldn’t respond to the customization settings I was setting in the Drupal backend.
2. Do some more research, find excitement about the YUI editor. It seemed promising, but ultimately failed to allow image uploads despite hours of fiddling.
3. Turn to the other oft-mentioned option, FCKeditor. After a few minutes of fiddling supported by the included readme.txt, it’s actually working and uploading images easily. Amazing!

Not everything in FCK is customizable through the web, but that’s fine. It’s probably easier to comment out a line in fckeditor.config.js, anyhow. I started with the DrupalFull toolbar & took off a few things we don’t want.

Since versions are so often key with these things, I’m using Drupal 6.6, fckeditor-6.x-1.3-rc3 (the drupal module), FCKeditor_2.6.3 (the javascript bit).

Including a reasonably debugged WYSIWYG in the Acquia distribution would get me to take it for a test drive next time around, because this cycle is such an amazing waste of time whenever I go through it.

Because rsync 3 is one of if not the only OS X backup solutions that actually gets all the possible forms of metadata, I’m in the market for a backup solution that uses it. The current candidate is rsnapshot, a 6000 line perl program (!), conveniently located in macports, that wraps rsync to do smart backup things like keep snapshots via hard links. Thanks to the O’Reilly Backup book for pointing me to it.

• It’s pretty easy to set up to run locally. The main trick is that the configuration file requires tabs. First time in ~10 years I’ve had to turn tabs back on in vim (:set noexpandtab).
• For OS X, we want the magic -aNHAXx --fileflags --force-change args to make rsync behave properly with all the metadata.
• OS X has a weird directory structure, so if you try to backup /etc you just get the symlink that is to /private/etc, or if you try to exclude something under /var, you miss it because it’s really /private/var/bigdirectory. Buzzkill.
• Lchown.pm is necessary for symlinks to have the right ownership in snapshots. CPAN’d.
• Running things automatically on OS X as the privileged user is a bit odd, lacking as it does a traditional root account. Sudo does nicely, with NOPASSWD: /opt/local/bin/rsnapshot
on the backup machine. For the clients we need appropriate ssh settings, with some tricks to run sudo on the remote machine. Getting this running took a while, since I missed the fact that running rsnapshot via sudo on the backup machine meant that rsync would try to use root’s ssh key, not the backup user’s — fixed this with the -i arg to ssh.
• Restoring backups is just a matter of copying them from the appropriate snapshot dir, probably using rsync & the same arguments rsnapshot uses (easily extracted from the rsnapshot log).
• Multiple servers are done serially. If you wanted to run backups in parallel, you’d need one configuration file for every server you’re backing up, and they each need their own snapshot_root, logfile, and lockfile. For my installation, this’d be more trouble than it’s worth.
  

Another issue that temporarily gave me pause, now that we’ve got everything backed up with presumably correct permissions, is that our off-site backup procedures involve creating tar files & encrypting them. Fortunately, backup bouncer shows that OS X tar gets all the important stuff right, though it’s no rsync v3.

Thanks to MSCPA, I finally tracked down the Governor’s press release which, at first glance, has a reasonably clear description of the regulation’s intent. Also came across an analysis by Beth Israel’s CIO, a positive blurb from a Maine consultancy, and a brief mention by a MA payroll company.

As for me, I still need to do my official audit of our procedures vs. those specified by the regulation.

We last looked at this a few months ago, but have been revisiting it to come up with something a little more robust. Notes:

• there’s a good overview of using PHP with FM at the sixfriedrice blog.
• the API for FileMaker’s PHP interface is available at http://YOURSERVER.URL:16000/docs/PHP%20API%20Documentation/index.html
• I’d missed this last time around, but accounts & permissions are a little funky. fmphp needs to be added to the Extended Privileges of the database you’re trying to get to, and must have the same privilege set as the account you’re connecting as.
• The solution we settled on is a CLI PHP script running hourly, checking for mail to send. Launchd would be the logical way to do the scheduling, but always drives me nuts. Fortunately the server in question has cron set up (so much simpler!)

The code we’re more or less using:

#!/usr/bin/php
<?php
set_include_path(get_include_path() . PATH_SEPARATOR .
  '/Library/FileMaker Server/Web Publishing/publishing-engine/php/lib/php/');
require_once('FileMaker.php');

echo "PHP email-sending-script, running at " .
  date('m/d/Y H:i') . "\n";
$layout = 'Outgoing_Email';
$fm = new FileMaker('Layout Name');
$fm->setProperty('username', 'your filemaker username');
$fm->setProperty('password', 'your filemaker password');

$findCmd =& $fm->newFindCommand($layout);
$findCmd->addFindCriterion('Sent_Flag', '< 1');
$result = $findCmd->execute();
if (FileMaker::isError($result)) {
  if ($result->code == 401) {
    exit("No emails to send.\n");
  } else {
    exit("trouble: " . $result->message . "(" . $result->code . ")");
  }
}

$records = $result->getRecords();
foreach($records as $record) {
  echo "To: " . $record->getField('Recipient') . "\n";
  echo "Subject: " . $record->getField('Subject') . "\n";
  $headers = array(
    "From: filemaker@example.com",
    "MIME-Version: 1.0",
    "Content-type: text/html"
    );

  /*
   FM helpfully encodes < and >...
   */
  $body = preg_replace('/&lt;/', '< ', $record->getField('Body'));
  $body = preg_replace('/&gt;/', '>', $body);
  $rc = mail($record->getField('Recipient'),
       $record->getField('Subject'),
       $body,
       implode("\r\n", $headers)
      );
  if ($rc) {
    $update = $fm->newEditCommand($layout, $record->getRecordId());
    $update->setField('Sent_Date', date('m/d/Y'));
    $update->setField('Sent_Time', date('H:i'));
    $update->setField('Sent_Flag', '1');
    $result = $update->execute();
    if (FileMaker::isError($result)) {
      exit("trouble updating the database after sending email: " .
        $result->message . "(" . $result->code . ")");
    }
    echo "Mailed!\n";
  } else {
    exit("Mail didn't work.\n");
  }
}

Update: I’d forgotten to check the flags that had been missing from the macport version last time around, but they’re all there. Now, running rsync -aNHAXx --fileflags --force-change /Volumes/Src/ /Volumes/rsync3test passes every bbouncer test! Cool.

Original post: On Leopard (10.5.5) using rsync 3.0.4 from macports, a few more tests are passing backup bouncer 0.1.3 (compare to my June post):

$ sudo ./bbouncer verify -d /Volumes/Src/ /Volumes/rsync3test/
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... ok (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok

I’ve spent a few hours this weekend poking at Superstruct, an ARG set 11 years in the future. The game posits that 5 interrelated challenges are on track to wipe out the human race in another 20 years, and it falls to the players to assemble existing social structures in new ways (hence the name) to overcome those problems and extend humanity’s existence.

Observations:

• I have a habit of thinking about medium-to-long term societal challenges, so the premise works for me.
• The first task, developing a profile of what your life will be like 2019, is pretty interesting. Although I’m used to thinking in these directions, having such a specific task is a different kind of exercise. Developing your own narrative before spending time exploring other people’s content is an interesting sequence, too — as I’ve been getting more into the game, I’m starting to appreciate the narrative challenge of 5000 users all telling their own stories that disagree with each other & with the official narrative in various ways.
• The main site is not very well done. It’s always logging you out, has a crude UI, and an absence of content navigating tools. However, people are working around those limitations on 3rd party resources, such as their own blogs, wikia, and on custom-built services like search and a player directory.
• The official story is largely set by a handful of video briefings. In trying to get up to speed on the story, I didn’t find that to be very effective. I think I’m really looking for the wikipedia of 2019 (or, of 2029 looking back, to mitigate POV issues).
• The game’s supposed to run for 6 weeks, and it’s close to being halfway through. Check it out now if you’re interested.

Will I spend more time with Superstruct? Maybe, maybe not. On the plus side, it’s a bold idea, and I’m curious what kind of projections and plans people can come up with in a game setting. On the minus side, I’ve got a whole bunch of real-world contributions I’d rather be making in my spare time.

Would you believe that a branch office of a certain unnamed organization hasn’t had a firewall (or even a router doing NAT) for close to a year? You could configure the printer from across the world and everything. That is, until the other day, when I got this little guy installed. Questions I had to answer in this project:

1. Which soekris? Current needs are reasonably simple firewalling for a smaller office, so the 4501 answers nicely. Would consider 5501s for an office with more traffic / servers.
2. What media? 2 GB CF cards are cheap as dirt now, I remember running OpenBSD on systems with smaller hard drives… the lower heat / power / footprint and higher reliability of CF is very attractive for this application, compared to a hard drive.
3. What OS / distro? Initially, I’m going with what’s basically a default i386 OpenBSD installation. Commonly used for firewalls, and given the cheapness and reliability of CF these days, a generic setup should work for now. Once I’m comfortable with how things are running I’ll look into more options for minimizing writes, and possibly mounting some or all of the FSes ro. The most promising reference I’ve found so far is Michiel van Baak’s guide.
4. What style of installation? Having serial access seems like a much better idea than just blindly booting CF cards, for troubleshooting purposes if nothing else. This also flows from the previous question, if I’d settled on a flashdist type approach, I’d be writing via a CF card. As it was, I got a keyspan USB-to-serial connector (drivers for OS X and linux!) & a null modem, and talked to the 4501 via screen /dev/cu.KeySerial1 19200 from my laptop. I then booted the Soekris from a convenient Debian server, with some helpful tips from here.

Stumbling blocks I ran into:

• The whole TFTP booting situation was tricky due to a missing ‘next-server’ directive in my dhcpd conf file, which I fixed thanks to this.
• After that, I got hung at

  
  boot> boot bsd.rd
  booting tftp:bsd.rd: 4780308+874136 [52+178240+163973]=0x5b821c
  entry point at 0x200120

  which was fixed by doing

  
  boot> set tty com0
  boot> stty com0 19200

  before booting bsd.rd (thanks to google’s cache of an mailing list discussion for that).

As any of you who’ve tried it know, it’s not that hard, though it did take a little trial and error to teach myself based on web howto’s. I ended up referring to this, this and this.

Why bother? It really is much nicer when your cables are the right length, so you don’t have unruly spools of unneeded cable everywhere. After a recent rewire, my server closet was a real cabling disaster. Also, it’s way more cost effective if you’re talking lots of cables.

Lessons:

• you don’t really need a cable tester. At least the simple kind I got wasn’t even that helpful — it gave a passing score to a cable that wouldn’t reliably work. Just plug it into your network & see if it transfers a big file properly. (If I’m missing something here, speak up!)
• a good wire cutter is absolutely essential. My crimper has a decent little cutter built in that’s just the right size, which is handy.
• T568B is what you want unless you have some weird situation
• Never realized how I’ve taken patch cables for granted, even in my days of installing servers. I guess Willy must have made sure we had a common supply, but I can’t really recall.

Networks Unlimited just sent out a note (thanks!) about the Mass Office of Consumer Affairs’ new
Standards for The Protection of Personal Information of Residents of the Commonwealth, aka 201 CMR 17.00: M.G.L. c. 93H. It outlines the responsibilities of anyone who gathers personal information on Mass residents. At a glance, they look pretty reasonable. From the intro:

Every person that owns, licenses, stores or maintains personal information about a resident of the Commonwealth shall develop, implement, maintain and monitor a comprehensive, written information security program applicable to any records containing such personal information.

It’ll be interesting to sit down with this & see how our policies & procedures match up.