Every time I have to look at plain ol' javascript it feels like my eyes are going to bleed. Regardless of whether I wrote it myself or whether another developer wrote it.
On the other hand, jQuery (still not the Miss World of programming languages..) is far easier on the eyes.
Also, for most apps, the performance impact isn't all that noticeable since most of the performance hit happens due to reflows and not because of jQuery. Plus, the Google CDN jQuery is pretty well cached in most browsers.