How many people, numbers wise, do you think were discussing particle physics? Yet, there we are: the Web was devised to foster such niche discussion and make the information in there easily findable and sharable amongst researchers. It just turned out that what Bernes-Lee & Cailliau came up with, could be applied to what the general public likes to talk about, too.
So I don’t believe that innovation in web and search technologies is keyed around what consumers want. Rather, it’s the other way around: more demanding use cases drive innovation, some of it can be repurposed and then gets fine-tuned thanks to the bigger R&D budgets that come with mass market application.
But the seed of innovation lays in the niches.