The comparison is not b/w fusion and a hypothetical waste free source of energy. Its b/w fusion and fission. The waste products of fission are much more dangerous, expensive to handle and we still haven't found foolproof, effective ways of disposing them.

OTOH, danger from irradiated materials (whatever that is, this is the first time I'm hearing this TBH) doesn't seem very pressing. I highly doubt any of the irradiated stuff would have a half life of millions of years.