I'm all for making performance a priority, but the IE team found a way to implement pointer events without sacrificing performance. Surely they aren't uniquely capable of this engineering feat.
In the long run, shackling the web to the iPhone 1's touch input scheme seems awfully short sighted. I don't know if IE's pointer event model is the best alternative, but touchstart/down/up/stop definitely isn't.