Certainly the generation of radioactive material in fission plants was considered potentially fatal all the way back to Fermi, who said way back when:

> It is not clear that the public will accept an energy source that produces this much radioactivity and that can be subject to diversion of material for bombs.

However, the nuclear fission industry has arguably shown that it can technically manage radioactive wastes without undue harm to the populace. Fission plants have killed up to 4000 people while producing 5% of the world's energy. Fossil kills 4.2M people/year at 84% total energy. The math suggests that nuclear fission has been highly successful. But the public largely still doesn't like it.

Fusion indeed has a better going-in position. Potential complications involve how hard/complex/expensive it is to realize the engineering challenges.

Aneutronic fusion plants, like fossil plants, will likely also have an EOL from a thermal creep, corrosion, cracking, etc. POV. Normal fusion plants will definitely have a EOL from neutron doses to structural materials.