Performance is in ms, see
http://caffe.berkeleyvision.org/performance_hardware.html
Lookup the testing numbers, 500 images / sec.
Character-based models are way simpler, you can use test very easily using the original link to the model.
They are not easily gamed either: they learn features as convolutional filters over the characters. These filters are more powerful than ngrams, as complex as the filters you may have seen for images.
Now, about gaming the nets, there's an issue, which is the same as for image-based CNNs: some combinations of letters (or pixel) do exist that do not make a difference to the eye, but push the classifier into the wrong class. However, in order to find these combinations, having a first-hand on the underlying neural net and its weights is mandatory.