For a Deuterium-Tritium cycle, half of the fuel will be in the form of Lithium because you will breed Tritium from it in the reactor blanket. Lithium is a relatively rare element in the universe because it gets burned up in stars. It's not clear that the Lithium resource is bigger than the Uranium-in-seawater resource or the terrestrial Thorium resource.
Another issue is thermal dissipation. It is already challenging to get rid of 2-3 GW worth of heat from a nuclear power plant. You could dissipate that heat by evaporating a modest amount of water, but that water could then condense and cause dangerous fog and icing on nearby roads, for instance.
If fusion plants are scaled much bigger, say 50 GW thermal, getting rid of the heat is more difficult (most practically it could be disposed of in the ocean.) The plants have to be cited further away from cities, power has to be sent further, all that adds to the cost.
It would be beautiful if fusion could support human life beyond the "frost line" where small bodies are rich with volatiles such as water, ammonia, carbon monoxide, etc. Imaginably you could cut up something like Pluto into small ringworlds that provide more living area than the Earth.
At some distance solar energy gets weak but a practical fusion cycle (possibly D-D based) might make it possible to hop to the nearest star from comet to comet over a time scale of 20,000 years or so.
The trouble is that people used to the Oort Cloud lifestyle probably won't be too interested in stars if they've managed to live that way for 20,000 years.
What you have to beat is both current-day sources of carbon neutral energy and also emerging sources. For instance if you drill a hole down a few miles into bedrock you might be able to withdraw heat for a century or more, possible advanced nuclear fuel cycles involving plutonium and thorium, space solar energy, etc.
Much research in fusion energy is driven by national security or pride interests. For instance the laser fusion facilities run by the US let the US develop nuclear weapons without nuclear testing. Projects like ITER are cultural "stretch goals" like the International Space Station; a lot of risk is involved but maybe there is a reward in 20+ years. Some people have always thought they could make an end run to a fusion reactor but so far it has not panned out. That doesn't mean it won't for somebody.
Because of that, I believe that if fusion were to become viable, at least in my part of the world, it would have to start with a serious shift in worldview. People would have to become politically astute enough not to fall for stupid Trumpian divisive tactics. And, we would have to adopt a stance in which the far future is as important as our present.
I really wonder what kind of advancements a species like that would want to make. My suspicion is that a species like that would invest much of the savings into completely eliminating the very concept of poverty.
I know that eliminating poverty isn't a terribly sexy answer, but I think it's the most likely big change.
On the other hand, we would really need that shift in world view. Without it, global warming would be avoided, but I fear we would destroy the world even faster than we do now.
Free energy makes anything produced automatically free. That means we could afford more disposable junk, more and bigger houses, more and bigger cars, likely even flying ones, with many flying supersonic.
Increased mobility would mean less pristine wilderness (if energy is free, any group of people could afford to do a mini burning man anywhere in the States)
Also, we would also scrape the ocean floor for minerals, destroying the ocean.
Finally, I am more cynical w.r.t. poverty. We already could have solved it, but didn’t. Why would that change?
Not insulting them would be a good place to start.