People aren't as entertaining as they think they are when they're drinking, and not being drunk means you really need other aspects of the experience to fill in.
You also have the problem that people get suspicious of people hanging around not drinking alcohol.