Have you heard about the inverted index? It is the corner stone of all databases and information retrieval systems. Your question is quite fuzzy so it is hard to come up with a more precise answer.
Something to always remember: if it is valuable, charge for it. If it is really valuable, you can spend all kinds of hardware on it. Give each customer their dedicated instance and rinse and repeat the strategies above.