My experience is totally opposite. I've found Google voice search very effective, even in a crowded bar. When I listen to some the recordings that were successful[1], I'm frankly amazed at what a good job it does.
The most hilarious clip I found was one where Google had transcribed the request as "bbbbbbb bbbbbbb bbbbbbb" and on playing it back, I heard the sound of my alarm clock going off with a steady beep, beep, beep...!