You're right about the difference that lowering your body mass makes. It's the easiest way to drop the weight that you have to pedal around. I have a cyclist friend who loves to claim "I just took a $1000 shit," the joke being that that's how much it would cost him to get a $poop_mass lighter bicycle.
However, this doesn't apply to racers. If it's going to speed someone up by 2%, and he's got the money, and he wants that last tiny advantage, he may just drop the cash on a $1500 wheel set.