It does NOT need cloud connectivity and all of these devices should be able to communicate locally to a matter or zigbee hub or over Wi-Fi without Internet directly to my server. That is the actual problem. We should not let corporate greed stifle innovation by saying new features are pointless because a company then can try to exploit it for further profit.
But on how we do that, my opinion seems to differ.
I postulate that it's hard (expensive) to do what you suggest: finding people to build that for every customer, while not increasing support costs is tricky today.
Just ensuring your personal computing device (a phone, laptop or server running home assistant) can see and talk to your device is a hard problem (which is why the go to solution is poke a hole in your router fw by pushing data to a server, and have mobile app only talk to the cloud).
Can we, as a software development community, come up with an approach that makes this easy to do for local first but remote enabled?
Some things that are already wildly out of control cannot be fixed from within. We can only hope that regulation and government influence could stop the waterfall. Or, a good old fashioned tea party (if that would even have the slightest effect nowadays).
So I think it's either increased government regulation, or IT crowd working to simplify development of local-first/mobile-supported applications for any type of a connected device and client. I don't really think this will come from a community, but a push to standardize on a couple of protocols, API formats, how apps can talk to the same API locally and over the internet and such — those are things that could really be done once (or at most a couple dozen times, for everyone's favourite framework and language :)), and then there won't be an "expensive" excuse for companies.
Or, rules can mandate that, when it will become cheaper because companies will join together to bring the price down (like they did with Matter).
The worst washing machine I've ever had is my current one, and it isn't even a "smart" appliance. It has just enough software to be worse than my one with dials for everything.
At least we are seeing actual, tactile buttons start their return in cars!