This argument has been applied to many rights in the Constitution, and the courts have consistently held that any right that can be cast aside just because the government asserts an "emergency", regardless of type, is effectively a right that does not exist. The job of the courts is not to minimize risk to the population but to preserve their rights, and they mostly do that.

As a practical matter, this is on the right side of history. There are myriad examples of governments stripping basic rights from the population in the name of public safety or some other contrived (or real) emergency. It virtually always ends poorly for the population because governments consistently abuse this latitude when expedient for whoever is in power.

Would you have the courts remove all rights from anyone the government deems a "terrorist"? Or give Trump the power to suspend elections because pandemic/reasons? It is exactly the same argument. Giving people strong rights carries risks but also powerful benefits.