You miss my point. We can all agree that being late is bad. But why villianize late people as selfish, when in reality, most are just absent minded.
Chilling out is a preferred course of action. Also, hundreds of millions of people are parts of the cultures where it's absolutely normal to be late.
Not to mention, those inconvenienced by the late party also have expectations they have to meet, and so on. A professor who is 20 minutes late but still wants to lecture for a full hour can make you late for your next class, for which you are punished, not the previous lecturer.
Should we chill? To a certain point. But after that point, it's our lives which are negatively affected, and being annoyed at someone who is making our lives harder is hardly unreasonable.
I'd prefer a person who is constantly late than a one with your attitude.