There's an old saying that the Constitution is not a suicide pact. I think that applies here.
Yes, if the government--at any level--was forbidding people from meeting for religious reasons for an indefinite period simply "just because," that would be a violation of our freedoms and something to be gravely concerned about.
Restrictions on religion not based on the religious belief itself have been allowed since the founding of our republic. You can't have a religion where a "sincerely-held belief" is that one parishioner in attendance gets murdered. Or that smashing in people's front doors is a sign of holiness. Religious buildings, while exempt from property tax, must still comply with building, electrical, plumbing, and life safety codes.
In this situation, given that it is content-neutral--all gatherings of size are prohibited--and time-limited and in the furtherance of a major social interest (inhibiting the spread of a brand new virus), I think it withstands Constitutional scrutiny.