The whole thing got completely out of control and we lost the transparency of the web. We could open the source, see how it was made and learn some new tricks. The indexers like search engines could index and understand the content only by reading the data and metadata of a site.
Now is such a black box, that if you need to index for instance, you will need to use a browser to render the whole page before you or any automated process can understand it. Search engines now need to have ML based image and language recognition just to understand the contents of a site (this is nuts). The free trade of information is much more expensive and the web is turning into a black box day-by-day..
The whole "appization" of the web for which the biggest leader is Google, is leading to the fall of a medium of information that have so much potential.
Sorry for the rant, this is not the best place, but i cant see any move that can change the course of this medium serving mostly the big guys, leading to more concentration of power and resources.
Thats why i've think the only answer that can turn the table, need to start fresh, because the game the web is in, is already rigged, and i think that it has a lot to do with the design, not just the politics, strategy and the big money.