Masking during a pandemic is never just about individual risk. Collective action has an exponential effect on prevalence, so the real risk goes down by far more than 5%.
Not taking action to reduce risk of illness or death is irrational.
There's a trade-off between doing things you enjoy (or don't enjoy) and the risk of death, and it's a personal trade-off, not something that can be assessed rationally.
But those who are the most likely to be affected i.e. older and vulnerable people would.
So in fact whether you wear one or not is far less relevant.
You misunderstand the utility of masks, where it is significantly more an infected wearer of the mask who protects others by their wearing. So, being infections before you have symptoms is the problem, which is why people should - as is standard in Japan, for example - wear masks in public whenever they feel unwell or feel a 'cold' coming on.