tens of thousands of people in the USA die every year from influenza. This does not cause most of us to be "scared" of influenza. Why not? Because (1) the individual risk is vanishing low except for a relatively small high risk demographic (2) the public health aspects (i.e. overloaded health care systems) are generally easy to control because it is not extremely infectious.
(1) is and always has been true for COVID, but (2) remains unresolved at this point.