Being new to this space, I would greatly appreciate your sharing your thoughts and any documents that i can use as reference.
I think about this subject a lot. Three key innovations we'll see in the "next wave" of search engines that flood in to replace stinky incumbents (won't name any names here)
1. New user interfaces. My effort, Glorp, attempts to transition from the "link dump" style of results page to something akin to Instagram. Faster to scan, fewer surprises, more friendly for kids and old folks. AR/VR will have some unusual needs. Audio - Alexa and Siri. We need search (in different forms) everywhere. [1]
2. New business models. Privacy is in, advertising is out. That changes everything about the model. We'll see engines with tons of niche features, like link collections, recorded search journeys, curated subject matter experts, family accounts with parental controls, and, most importantly, monthly fees.
3. New search mechanisms: ML embeddings. Search by image similarity, video similarity, subject matter similarity. Machine learning models transform different types of content into convenient embedding vectors. BERT gives you a few hundred floats to represent the meaning of a sentence. ImageNet, the same, for images. Once you begin to build search tooling around the notion of document embeddings, rather than just "input text," we'll be able to help users discover things in a much broader set of scenarios. [2]
Hit me up, email in profile
[1] https://glorp.co/Search/search%20engine%20user%20interfaces
[2] https://glorp.co/Search/search%20engine%20vector%20embedding...
I doubt Google would really use them.
It was not very practical as a generic search engine, but I found some very interesting links following the maps of related terms. I thought that kind of intuitive user interface would be improved and become popular, but they never did.
In the beginning there was just string matching, word stemming followed, then we had synonyms and are getting more towards NLP for getting the user intend. Also, if the communication with the engine is via a text box or via a voice command or a live smartphone picture (for augmented reality) does not really matter.
So, what can change is just the "smartness" of information retrieval at the backend and the ranking factors (from freshness to PageRank to whatever).
Given the fact that the market power of Google actually shapes the internet by introducing their best practices (like AMP or schema.org) on webmasters on a large scale, I doubt that there will be one company with the big breakthrough in information retrieval like Google was. Maybe this field is now more or less saturated for the majority of use cases that it will only evolve slowly and not revolutionary.
+ Tree-based navigation (go back a page and overwrite your cached future? no thanks)
+ some sort of warmer/colder haptic feedback could be cool if there's a convergence on results for specific terms
+ some useful topic overview of things I searched in the last 1 year, 2 years, 3 years, etc. See where my mind has been.
https://twitter.com/ianbremmer/status/966677755424313344?s=2...
Not sure I (or pretty much anyone else) care how it does that.
So the imbalance today exists because the search engine uses AI against humans.
The next phase will be (or should be) AI against AI.
The human UI of the search engine will not be used by humans but by an AI representing the human.
If you follow this paradigm, you can see how powerless the centerl search engine becomes.