Its actually not that bad, but it really depends on your browser. I've been throwing together (meaning not at all finished) a tower defense game using it (
http://www.willarson.com/code/processing/ptd.html), and it plays amazingly quickly on the current release version of Safari, and plays just slightly slower (but still perfectly, just a different in how quickly frames are processed) in the beta 5 of FireFox 3. On the other hand, in the release version of Camino it runs horrifically.
There are a ton of issues with it, in particular not working on IE at all, some features needing beta browsers, etc, but I don't think that speed is going to be one of the more important limiting factors. I imagine the demo you looked at was doing something pretty crazy, either visually or in the code ;)