Edit: in case the sporadic downvotes indicate some confusion, let me elaborate.
Certainly there is a common argument which involves comparing artificial radioactivity to natural sources. The infamous "banana equivalent dose" is an example of this. But that's fundamentally a quantitative argument about accepting small doses. It's not a general argument for neglecting the danger of nuclear waste in general.
The argument that you should, say, not worry about the dose you get from the americium in your smoke detectors, or the ocean contamination from Fukushima, can be made based on this. That argument is fundamentally sound. But it does not imply that you should go swim to the bottom of a spent fuel pool and hug the canisters, or that it would be a good idea to have a play in the mud next to Chernobyl.
There's nothing fundamentally wrong with making similar arguments for coal. For example, if the amount of mercury contamination in seafood was small compared to the amount of mercury naturally present, it would be quite sensible to argue that coal is not a major problem when it comes to mercury pollution of the ocean. The problem, of course, is that it's simply not true: the natural level of mercury is pretty low, and the contamination from coal power is substantial. When the FDA recommends limiting the intake of seafood for young children and pregnant women, it's not because of the naturally occurring levels, it's because of contamination due to human activity, of which coal is the largest part.
One extreme is the "ideal dispersal" leak, like the Fukushima leaks into the ocean. In this case, the radioactive material does not "clump up" enough to be dangerous -- it's not that some cubic meters of ocean are fine and some contain deadly, deadly poison, it's that each cubic meter of ocean contains the same minuscule amount of radioactive material, too low to be a threat.
The other extreme is the "immobile clump" of radioactive material, like the Elephant's Foot at the heart of Chernobyl. Here you have enough radioactivity to cause serious harm or death if you get too close for too long. It's scary, but it's also pretty stationary. It's scary like a hole in the ground. You stay away, you warn people not to fall in, but it's not going to creep up on you without warning, like a sinkhole in Florida.
The really scary kind of radioactive waste isn't either of these two extremes, but a dangerous middle-ground: radioactive waste which is clumpy enough to give off deadly amounts of radiation, but mobile enough to get everywhere. You occasionally get this from reactor leaks (that's the primary reason for the exclusion zone around Chernobyl; on the other hand, they could never find any evidence of this sort of leak after Three Mile Isle). You also see it in non-power-plant nuclear accidents, like the Goiânia Accident, which involved a radiation source used in medical imagine.
You also worry about it in coal ash, because the ash can be blown and scattered but is clumpy enough that the radioactive particulate doesn't disperse.
And that's true even when you account for the risk of nuclear power plants and spent fuel. Coal ash is produced in gigantic quantities, and often stored in an open environment, while spent fuel is vitrified and far easier to contain.
https://www.scientificamerican.com/article/coal-ash-is-more-...