I really like obscure search engines, because their (comparatively :P) tiny indexes mean they'll return significantly different data to what say Google or Bing would return.
If there's one thing I would definitely vote for with projects like this, it would be... an API. In a world where virtually every search engine is built around the idea of controlled, non-open access, a predictable, documented, free-access API would be a breath of fresh air, and you'd get people seriously interested in this even though it's small. (You'd definitely need rate limiting though because there are always the hangers-on with the low IQs. >.>)
You may (or may not, no idea) also be interested in the Common Crawl corpus - http://commoncrawl.org/the-data/get-started/ - which contains a few PB of data collected over the past few years (sorted by date). Access via HTTP is only moderately involved: read the line immediately above "Data Format."
Of potentially much less interest is a 2012 80TB snapshot Archive.org are (or at least were) offering; they seeded their indexer off the Alexa top 1m and boxed up whatever came back. https://blog.archive.org/2012/10/26/80-terabytes-of-archived... It's apparently only available on request (I expect a search engine would definitely qualify, especially considering the data is 4 years old now - although I definitely want to see what's in it if you do decide to chase this - serious :P).
Finally, search for "cheetah project" - the 2nd top-level result is messed up, apparently the Chinese (Unicode) got scrambled. Just a note; you may already be aware.