Another proposal: a standardized search protocol over HTTP that can be implemented by most web servers and a DNS-like system of search indexes that use websites' built in index query endpoints (rather than crawling). Essentially decoupling search and advertising by building a search function into the internet standards.
Perhaps commercial products can improve over the quality of results, but simple text and keyword search can work this way.