I see it as a statistics problem where we need to literally choose where to draw the yes/no line:
If COVID was 100.0% deadly and the vaccine is 100% effective, (think rabies) would we be justified to force the vaccine? Sounds like an easy yes? Even here, some would want to die as it is "God's will."
If COVID was 0% deadly and the vaccine is 10% effective, (you are vaccinating to protect other mammals or some rare species of life) how could we justify any force?