>... compared to google search which you'd never be able to host locally
That's something I haven't thought about for a long time. How big is the Google index, would you say? Petabytes or exabytes, or am I vastly overestimating?
That's something I haven't thought about for a long time. How big is the Google index, would you say? Petabytes or exabytes, or am I vastly overestimating?
If the index is less than petabytes then it's impossible to serve many long tail queries even just based on text content. IME those queries _have_ gotten progressively worse results the last few years, so maybe the techniques have changed, but when the money printer is "search" and a petabyte of data is a fraction of a full-time engineer in cost to store, I doubt they would have cut costs that aggressively.
Accodrding to Google:
https://www.google.com/search/howsearchworks/how-search-work...