Yeah it would suck, but if Ford F150s were selling for $100,000 because drag racers were buying them up, I don’t think I’d mind.
We often have used GPU's for around the price range you're looking for. They won't be fantastic, but probably an upgrade for a 9 year old machine.
Edit: I should also add, a lot of the really small scale shops also don't list 100% of their inventory online, with the benefit that it doesn't get scalped up like everything else.
Probably the most critical usage of GPUs is scientific simulations for biology/medicine, climate, space, etc.
Modern computers have allowed these 'forever existing' technologies to scale to our population growth. Try building freeways without modern computers, or performing modern medical interventions without them.