>I will pay a lot of money to buy hardware and software that actually shifts back towards programmable stuff that I actually own and can do what I want with.
So that's exactly what would happen. Those who have the bucks will buy hardware and do customizations, while the rest will buy standard display, and camera and suffer the latency, privacy and personalization problems.
I don't see it as difficult. Only thing is right now the hardware prices have been moving down. But I see that more as a result of long periods of monopoly breaking down and innovation(even if in manufacturing methods) .