1) Massive and public damage done to the cause of liberty. Picking a minor, I wouldn't want to imagine the looks that someone protesting the Roe v. Wade overturning got if they tried to chant "my body, my choice!". There is firm precedent now for a generation that your body, government's choice. We're more likely to regret that than a few people hurting/killing themselves out of stupidity refusing to take a vaccine as the authoritarians start making more invasive demands.
2) Forcing someone to do something against their volition, followed up by the rather obvious point that the coercers were acting without full information or particularly good intent (see: "Why did COVID become so polarizing?"), will have consequences just from the bad blood it created. There are some angry, angry people out there now. People who have a very compelling argument that it is acceptable to coerce your political opponents and punish them in disproportionate ways.
3) It still isn't obvious that the all in cost/benefits of the policies were worth it factoring in 2nd order effects and opportunity costs. We're staring at nuclear war between the two countries with the largest arsenals right now. It might have been worth just ignoring COVID, letting people make their own choices for better or worse and focusing on not finding a diplomatic peace in the northern hemisphere. The nuclear war risk is probably higher than the COVID risk, although a lot more all-or-nothing. It was almost completely ignored because of the resources spent trying to keep unwilling people inside their own homes. Also the global economy overall is in a terrible place, the damage is incalculable but large and probably comparable to losing 1% of the population. There is a famine problem out there [0].
A well-meaning analysis and noting that 1% of the population could die doesn't really capture the trade offs here. 1% of the world is going to die every year anyway; doubling that is bad. Normalising, justifying and defending mass suspensions of human rights is, amazingly, probably going to be worse.
[0] https://www.who.int/news/item/06-07-2022-un-report--global-h...