I wonder if, rather than storing an "image" of a face, we store a sort of error correction from a standard model. This is akin to probabilistic compression, where you have a bitstream generator and then store a list of where that bitstream is in error.