You should, because you'll quickly realize that there are far, far more activities that potentially cause harm to others than there are laws that prohibit them. Also, vaccines and smoking have a history that are peppered with lawsuits about their legality, so as far as examples go, they are not the ones I would hold up as evidence of laws that people clearly think are constitutional.
The flu killed ~50,000 last year in the USA. Do we require all non-essential businesses to shut down and everyone quarantine? Is there some magical cutoff number between 50,000 (flu deaths) and 80,000 (COVID deaths) where we decide to shut down the entire country?
Also you're conflating "the government has the right" with "there exists a law." The government has the right to enforce the law, but there is no innate right in that law itself. If we overturned smoking laws tomorrow, it doesn't remove rights from the government, it removes laws that the government enforces.