All right, let's suppose for a minute that self-interest is literally the only thing to be considered.
Your personal risk preferences impact me in a variety of ways:
* my cost of living goes up, because damage control is much more expensive than prevention; someone has to pay for it, and dead people can't.
* my personal risk goes up, because resources I might need such as emergency doctors' time are occupied dealing with the consequences of your decisions instead.
* my quality of life goes down, e.g. I have to spend more time waiting in traffic jams while the bodies are scraped off the roads.
If you take your risks in a space I am forced to share with you, whether physical, societal or economical, your choices affect me.