The thing is when people around me are "ruining their lives" it does affect me.
Crime goes up, bankruptcy goes up, corruption in sports goes up, etc.
I agree that people should be given freedoms, but we live in societies and people aren't independent, disconnected, autonomous units.