What downsides are there to fusion power, assuming it's commercially viable in a couple of decades? What negative externalities are there that could make it prohibitive? Fossil fuel has pollution etc, solar takes up lots of space and is apparently also ugly, wind energy is ugly and makes noise (according to NIMBYs).

What argument could a NIMBY person have against fusion power, besides the big building in their back yard?