rlucas.net: The Next Generation Rotating Header Image


Don’t Bother with S3FS for Mounting Amazon S3 on Mac OS X (2014)

As of January 2014, it’s not worth bothering with the “s3fs” software for mounting Amazon S3 buckets on your local filesystem.

The idea of s3fs is simple and great: use FUSE (Filesystem in Userspace) to “mount” the S3 bucket the same way you’d mount, say, an NFS drive or a partition of a disk. Manipulate the files, let s3fs sync it in the background. Sure, you lose some reliability, but we’ve had NFS and SMB and all kinds of somewhat-latent-over-an-unreliable-link-but-mostly-with-filesystem-semantics software for decades now, right?

Well, forget it. s3fs as of January 2014, used on Mac OS X and against an existing set of buckets, is so utterly unreliable as to be useless.

First, s3fs cannot “see” existing folders. This is because folders are a bit of hack on S3 and weren’t done in a standardized, documented way when s3fs was first written. However, since then, at least two other ways of creating folders on S3 have gained currency: an older, deprecated way with Firefox plugin S3Fox, and a newer defacto standard way with Amazon’s own management dashboard/browser for S3. Whatever the historical reason, you can’t see the existing folders.

Second, although from mailing list posts, theoretically you can *create* an s3fs folder with the same name as your existing folder, and then its contents will magically become visible, empirically something rather different happens. A mkdir on the s3fs mount leads to the creation of a mangled *regular* file on the S3 dashboard. Now you have two “folders,” each of which is unusable as a folder on the other (S3 dashboard, or s3fs) system. Argh.

Finally, you might say, ok, fine, this will just make me use flat-level, non-folder-nested choices about my S3 architecture. (Leave aside for the moment that the very reason you want to use S3 is probably exactly so that you can have lots and lots and lots and lots (like 10^8+) files in a way that will cripple any reasonable filesystem tools that see them all in one “directory.”) However, even that doesn’t work reliably, as s3fs demonstrated today when it went into “write-only mode” such that I could create files locally that would show up on S3 but that subsequently would disappear from my local filesystem. WTF?!?

The unfortunate answer is: S3 is not a filesystem, and it was created by people who are smarter than you, and who have very craftily calculated that if you are forced to weave in the S3 API and its limitations into your application code, you will have a damn hard time ripping it out of your infrastructure, and so they are going to have you do just that. They do not want it to be used as a filesystem, and so guess what: you are *not* going to use it as a filesystem. Not gonna happen.

Say what you will, but our hometown heroes here in Seattle are no dummies. Embrace, extend, extinguish, baby. Not just for OS companies anymore…

(Yes, I know that s3fs is not an Amazon project. But it appears to be the community’s best attempt to put filesystem semantics around S3, and that attempt has been rejected by AWS’s immune system.)

Never Trust Microsoft Products On Mac OS X, Again

Something in OS X 10.7+ and MS Office 2008 for Mac has made the entire Office suite, apparently, stop respecting symlinks.

I know that Windows took until well into the 21st century to really deal with symlinks even halfway, but come on. It worked before. There are some directories into which I routinely save document files which are literally not reachable by click navigtation any other way but through symlinks.

If you click on the symlink in the “Save” dialog, it will change the name of the file you’re saving to the name of the symlink and generally fail all over itself.

Never Trust Microsoft Products On Mac OS X

In Microsoft Word 2008 for Mac OS X, if you open a “Find” dialog, and if it has focus, it will capture cmd-S (the universal “Save” command) and cause it to toggle the “sounds like” option in advanced find options. It will sort of capture cmd-W (the universal “close Window,” which will close the Find dialog). It will not capture cmd-Q (the universal “Quit”), which will quit the program and close all its windows, assuming your document has been saved.

Which you thought you’d been doing by hitting cmd-S. And then you wonder why your search results were so weird.

Thanks again, Mac BU!

LinkedIn CPC Ads behaving weirdly

This June/July, over at SimpleVerity, we’ve been running some very tiny, super-targeted CPC ad campaigns on LinkedIn to recruit trial users of our system. We’ve noticed the following odd behaviors, and I wanted to ask the Web if anyone else had seen this:

– Lower bids (bottom of the range) outperform higher bids on both CPC and conversion (learn more form on my landing page). Weird, but perhaps explicable; it could be that for our particular appeal, the “lower ranked” ad slots work better.

– Temporary “spike” in CPC, and BIG spike in conversion, right after a change to the campaign configuration. This is strange, because it seems to result not only in a change in how the LinkedIn users react (could be a result of the ad rotation algorithms after a change), but also how they behave once they get to my landing page. No theory for how this works.

– Auto-Optimize among campaigns seems to prematurely start making optimization decisions before reaching anything close to statistical significance. For example, I was running a campaign where the long-term equilibrium average CTR seemed to be about 0.9% (not bad, right?). I had 15 variations running. However, the auto-optimize algorithm started heavily favoring a few ads before all variations had received even 1,000 impressions. That’s just crazy; the ads being disfavored were about 50/50 bets at that point to exceed the average, yet the algorithm was “optimizing” them into oblivion.

My takeaway is that, at least when working with heavily targeted LinkedIn CPC ads, you have to stay very, very close to the “metal” and pay attention to oddities. Don’t assume that paying the highest bid (or even paying more than the bare minimum CPC price) will work better; relentlessly experiment with spend and bid. Also, don’t let auto-optimize kick in until you are comfortable that you are approaching significance on the expected CTR (at the very least, the CTR that you are hoping to “beat”).

TrueCrypt breaks after OS X 10.8 (Mountain Lion) upgrade

1. You run TrueCrypt as before, and once you finally remember your passphrase and type it correctly, you get an error: “the MacFUSE file system is not available (71)”

2. You look around and you still have /usr/local/lib/*fuse*.so etc., in addition to /Library/Filesystems/fusefs.fs

3. Your MacFuse control panel in System Preferences still exists (but the “uninstall” button doesn’t seem to work!). Also, the script which that button runs, /Library/Filesystems/fusefs.fs/Support/uninstall-macfuse-core.sh — that doesn’t work either (run with sudo).

4. You’ve tried installing “fuse4x” and “fuse4x-kext” from MacPorts, but still nothing.

What I did:

1. Remove MacFUSE manually, completely, using the list of installed files at http://code.google.com/p/macfuse/wiki/FAQ

MacFUSE file system bundle (/Library/Filesystems/fusefs.fs)
MacFUSE Objective-C framework (/Library/Frameworks/MacFUSE.framework)
MacFUSE C-based libraries (/usr/local/lib/*fuse*.dylib) and headers (/usr/local/include/fuse*)
MacFUSE preference pane (/Library/PreferencePanes/MacFUSE.prefPane)
MacFUSE project templates for Xcode (/Library/Application Support/Developer/Shared/Xcode/Project Templates/MacFUSE)
2. Remove /Applications/TrueCrypt.app

3. port uninstall fuse4x and fuse4x-kext

4. Reboot. (Might not be necessary but can’t hurt here.)

5. port install fuse4x fuse4x-kext

6. Reinstall truecrypt (you might need to option-right-click to bypass the security check). Note that installing truecrypt, I believe, re-links the TC binary against the fuse libraries that you just re-installed using MacPorts, so you end up with a different TC binary than before.

7. Run TrueCrypt.app again and hope that it all worked…


Don’t try to do date math in MySQL

MySQL (5.5) has incredibly broken date math semantics.

Subtracting two “datetime” types gives you a number which is manifestly not the number of seconds between those datetimes. (It’s also not ~ 10^n times that number for any n.)

So naive subtraction is broken. What about “timediff?” Well, you can use that function to compare two datetimes, and what it returns you is in the form “HH:MM:SS.” Nice. Seems to be some kind of useful interval type, right? If you decide you just want to do some averages on this result, you can then use the “time_to_sec” function, and now you’re computing some nice figures like the average time to click on an email.

…which, in keeping with the MySQL philosophy, sort of works until it (almost) silently stops working. What I mean is that the first time your date math gives you a figure like “883:15:19,” the “time_to_sec” function will start quietly giving warnings. If you turn on warnings (which really REALLY need to be on by default, here, MySQL folks), you will learn something like:

Warning (Code 1292): Truncated incorrect time value: ‘883:15:39’

Oh, great. There’s some number of hours one can put into a time value which is subsequently converted into seconds. What that number is, who knows. (Apparently not the author of the “timediff” function, because that’s what gave me that incorrect time value.)

Just give up and convert everything to an epoch timestamp. After all, who needs milliseconds anyhow? Other solutions include: use Postgres.

Quick Django Foreign-key Gotcha

When your Django app poops out with:

'RelatedManager' object is not callable

The problem might be that you’re trying to call up related fields via a foreign key field type’s auto-generated reverse-lookup attribute — but mistakenly trying to call it as a method instead of as an attribute that holds a query set. In code terms, you put:

tracks = cd.track_set()

What you meant to do was:

tracks = cd.track_set.all()

The Draft and Google

Only because this will officially be the first page in the Google index with the words of a bumper sticker I saw (and which, I can’t lie, resonated a bit):

“Draft by Wealth: Most to lose, first to fight.”

Hmm. Thought-provoking.

The “user” virtual file / workspace in Prolog.

When you’re first learning about Prolog, sometimes you’ll read books or tutorials that show you typing in “clauses,” and then immediately thereafter typing a “query.” If you use a visual or browser-based Prolog implementation, you’ll discover that there are two “modes” (my term), one for the input of the program / database, and one for querying. The query form is denoted by the prompt “?-” as in:

?- my_query_(Variable).

To me, it was somewhat confusing as to why and how I had to keep separate my program from my queries, since I was used to REPL-type interactive programming (e.g. irb for Ruby).

Later, in reading the SWI-Prolog manual, I saw reference to this odd snippet in the “Adding Arithmetic Functions” section:

?- [user].

:- other_stuff(x).

It appeared that [user] was changing the mode from query-mode to program-mode, and allowing me to define new predicates.

Well, close. It turns out the way to think about this (can’t seem to find it in the SWI-Prolog manual) is that the square-bracket notation is the “load file” shortcut, and Prolog comes with a virtual file known as “user.” When you query:

?- [user].

and then go on to get:


as your new prompt, you’re loading the “user” file which is like opening a handle on STDIN. (Not precisely, but close enough.) You can then type your program entries, /but you must terminate them with EOF/ (ctrl-D).

This bit of prerequisite knowledge would have saved me a lot of puzzlement and trial-and-error. A shame that Prolog, despite a nearly 40-year history of continuous use and usefulness, has such a high propaedeutic load.

“Getting It” in IT Security

Look, folks, PKI is hard to grok fully. But if your entire business is being a trust provider (SSL cert vendor), you should try.

GeoTrust makes their Root Certs (the things you download and install in order to tell your computer, “trust things from this authority”) available via an insecure connection: