If you see the various implementations of measures in different countries as large scale experiments, I think you can derive some useful information and answer your own questions.
There is maybe some point of diminishing returns in wearing masks, but what the limit is isn't clear. Sweden seems to agree with you and against general use of masks, although my trust in their approach is limited by their pretty terrible results: death rate of nearly 600/M against HK's 12/M.
Regarding increased risk of falling ill because of the mask, there is not much evidence to support that. That's not saying it doesn't or can't happen but while you emphasise a lack of evidence to convince you that masks are worthwhile at scale, there is even less evidence that wearing a mask will cause any other type of hurt that would cancel-out the benefits.
I personaly hate masks, they itch, are uncomfortable and they pollute. It's clear they are not a panacea. They are however part of the small set of limited practical measures that we can take to help slow the rate of infection (by limiting the exposure of others to the crap we exhale).
Hong Kong will be an interesting experiment to follow because I foresee that people will continue wearing masks long after everyone else. We'll see the long term effects this has on the 8M people living there. I'm sure it will be worthy of a few papers.