How about climate change? Meteors, supervolcanos? These are all things that have destroyed the majority of either humanity or life on earth at some point in the past. And then there's horrors of war, plagues, earthquakes, and tsunamis, which have the capability of killing double-digit percents of regional populations in short amounts of time. Terrorism has never killed even a tenth of a single percent of any population.
Sure, you can worry that terrorists /might/ get nukes or biological weapons. But unstable national governments already have these things. The number of times that the USSR and the USA accidentally almost started a nuclear holocaust via tens of thousands of superbly-engineered ICBMs is vastly more concerning than a bunch of poorly funded, poorly organised idiots that might set off some homebrew weapon.
There are vastly more impactful things that /have already proven to exist/ to be worried about and prepared against.