There will never be an all encompassing search engine to rival Google. I would argue, one could not even rival Bing.
The future of search, as with most internet companies moving forward, is all about mastering a niche. You may not be able to beat Google as an overall search engine, but if you dedicate your time and money towards one specific niche, say sports, then you have a fighting chance to beat them at that.
IMHO there are multiple ways to create a search engine: 1)Mastering search within a vertical e.g. sports, music, etc. and then expanding to other verticals. 2)Go after a limited popular area of the web and provide a more personalized search experience(this limits discoverability of less popular sites but there are ways to overcome this) 3)Enhance the results of existing search engines by providing context, personalization, privacy features,etc. (this relies heavily on existing engines which makes it subject to their TOS) 4)Come up with an algorithm that blows Google's or Bing's out of the water and VCs line up to give you money so you can build the infrastructure to make it a reality (I wish I had this too... sadly I don't)