If each one is armed with a few anti-ship missiles, a swarm attack from many different directions might be a real threat to a carrier group. You either waste a ton of ordinance taking them all out, or some portion are able to launch against you.
It's a serious threat as everything is low low cost and big on bang.
The US would be looking at defending itself from 100+ unmanned cigar boats each packed with enough explosives to make a serious dent at water level and all interoperating like a flock of birds, some drawing fire while others push through.
With that said, nothing prevents the U.S from imitating any tactics (including swarming by tiny boats and ships) that the Chinese use effectively against the U.S and its allies in a real naval war.
To name a very current example of my first point above in action 24/7, just look at the Ukraine/Russia war today. It has evolved in ways that very few people could have predicted just before or right after it started, with surprises on both sides.
To highlight how the same worked at the beginning of the second world war, partially prompting Churchill's own later phrase: the air defenses of the UK were thought adequate against enemies in 1939, until they were proven to not at all be so, and only improved through very deadly trial and error by 1943. France in 1940 was thought by nearly all powers to be more than a match against a German invasion (and indeed materially it was, on paper and stationary on the ground), until it was proven to not at all work as expected in just 6 weeks. All things always shift simultaneously in ways that can confound any expectation, particularly in situations as ferociously dynamic as large wars.