This is likely to be cyclical though. Fast cars[1], large amounts of processing power[2], access to cryptographic algorithms, are all things that started out expensive to be on the cutting edge, some still are expensive on the cutting edge, but then became more affordable for the consumer over time. AI has already had explorations on training models with limited resources. It's feasible, just has tradeoffs that will hopefully get better over time.

[1] A 1970 Corvette 427 has a 0 - 60 mph of 5.3 seconds (src: https://www.caranddriver.com/features/g15379023/the-chevrole...) and cost around $44k inflation adjusted dollars. You can buy a 2008 Nissasn 350Z Enthusiast that will do it in 5.2 (src: https://www.zeroto60times.com/vehicle-make/nissan-0-60-mph-t...?) for around $13k today.

[2] I'm too lazy to calculate relative cost / cycle in old warehouse computers vs phones but it's gotten _better_.