The biggest thing that would probably help is not changing the features to SIFT, but to make the recognition part using a classifier rather than the set of rules he's using.

Thus, instead of having to join edges into lines, and then applying rules about parallelism, etc., one could take a set of training examples and train a classifier (e.g., support vector machine) on it.

But I agree, cool demo.

It seems a license from UBC would be required to use the SIFT algorithm in a commercial venture. This may have been enough to dissuade the algorithms use in this case.