The majority of gamers might not need or desire better hardware, particularly if they have to pay for it. For that matter, game developers have very little reason to invest in games that require more capable hardware. There's the silly pretense now that people will use more capable hardware to play the same game in 4K, 8K, 16K, but the awful truth is that a lot of people would rather pay $45/month for 1 Mbps DSL than pay $50/month for 1 Gbps fiber, or would rather watch a terrible camera capture of a movie from the theater than wait for the Bluray to come out.
Consider the success of the Nintendo switch. The dock for the switch was a bit of a Jedi mind trick, just like the Wiimote was for the Wii. In particular, it changed the competitive psychology of game producers to think that any "big game" that came out for the Playstation or Xbox would also be released for the Switch. Selling the switch as a hybrid device stops gamers and games company from realizing it is what is... the beginning of the end of the TV-attached console.
The Steam Deck is a much more powerful machine for about the same price and it also has a very strong catalog. The Steam Deck can play Control natively and the Switch can't. Yet for the most part the industry is going to keep remaking the same old games and releasing them for the Switch, even most the innovative parts are going to be making games like Genshin Impact for midrange phones.
Sales of the 4090 are driven by prestige, the pleasure you can get out of saying that you own the very best. They are not driven at all by the reality that you can have anywhere near 10x the pleasure from a $4000 gaming PC than from a $400 portable. However a business that revolves around neural networks very well might be able to generate 10x the revenue from a 10x faster graphics card.
It's not a question of what "we" can afford but it is a function of what "they" can justify to their shareholders. I mean, the people in my town could afford 100% fiber optic internet, but the phone company can't afford to stop using the copper network when it can still charge people premium prices for subprime service. If we decided that $100 a month was a fair price for gigabit fiber and that price caps should be scaled accordingly, my telecom would quickly decide that it "can't afford" to offer DSL at $0.10 a month and would start tearing up that copper network pretty quickly.