There are currently hundreds of nuclear fission reactors in the world so if you assume a similar number of fusion reactors, you'll get thousands of metric tons per year of gold production. World gold production was ~3,600 metric tons in 2022 [2].
So ultimately, yes, it would impact the price, but it will be many decades before we have hundreds of fusion power plants producing all that gold, and so it's not going to saturate the market for a long time. It may never saturate the market if demand for gold continues to go up in the coming decades.
1. https://www.usgs.gov/faqs/how-much-gold-has-been-found-world 2. https://en.wikipedia.org/wiki/Gold_mining#Statistics
Plus, 20 years is really not very long for it to be completely harmless; just ask the world’s supplies of whiskeys that really only start to have any value after 10+ years.
Maybe you wouldn't use that gold for bond wires (but they are less used now and mostly copper AFAIK).