If google stopped ranking the most relevant content at the top, I would start using bing instead.
Regardless, Google is doing this to push at the technical limitations on apps like GMail or Google Reader, not Joe Shmo's burger shop.
IMHO a site that cares about the user experience would have been optimized to load quickly.
Heck, even if Google told people speed was important, and didn't incorporate any speed measurements into PageRank, I think it would have a positive effect.