Tell HN: HN Search isn't indexing anything newer than 5 days old | Hacker News Reader

Tell HN: HN Search isn't indexing anything newer than 5 days old

43 pointspetercooper14 comments

Not much to say really but for all popular keywords I've just tried the newest item is 5 days old. And none of the current front page items seem to be visible either.

For example, there are posts about Ruby almost every day, but..

http://www.hnsearch.com/search#request/submissions&q=ruby&sortby=create_ts+desc&start=0

View on HN

Sorry guys - one of the parsing scripts died and didn't get restarted automatically. Normally we would have noticed the problem earlier but we've been away at a conference this past week. The script is back up and the index should be up-to-date within the hour.

I would also like to add that it only does exact string matching. HN could use a better algorithm. The search could, say, even crawl the website linked. That would make HNsearch a much more powerful search engine, its advantage being the huge amount of vote data collected from an awesome community. Instead of Google's back-link information, instead of Aardvark's social graph nearness algorithm, HNsearch could be a search based in the level of intellect contained in the links, and discussions. All it needs is some sophistication in the algorithm, maybe some LSA, LCS, etc...

I don't understand why hn needs its own search engine. What are the advantages of using hnsearch compared to using google with 'site:news.ycombinator.com'?

...because Google doesn't offer site specific options such as sort by date/points?

Google do offer sort by date but it doesn't work terribly well IME.

You could also do a rough number range search ("100..1000") that might help capture higher value comments but in practice is probably too rough to be useful.

Reasons might range from the aesthetic: the "search language" of the Google Search Engine is imperative, not functional; to the paranoid: Google might datamine searches from YC company IP addresses; to the utterly practical (and perhaps likely): a toy search engine is a good way to know enough about search engines to guide and evaluate a YC company working in that area.

Then again, it may just scratch an itch as an inhouse tool.

Google would only search the discussion threads, and not the content on the website. What I'm pushing for, is we can easily extract a 'credible' subset of the web and only search that web. That subset is the set of links that HNers have posted to HN, sorted in priority by the number of votes they have returned.

Also, as danso mentioned, HNsearch could use some function of points and date, to rank the results.

This is probably a good time to give my thanks and appreciation to HN Search, however. I use it a lot and it's a very useful tool. I appreciate whoever does all the work on it.

Feel free to leave similar comments as it might give someone a smile while they're being hassled to fix this over Thanksgiving.. ;-)

I asked the Octoparts.

Thanks, I did e-mail and tweet the @HNSearch account as well but thought it'd probably do better here.. :-)

PG I love how you call the companies the "Octoparts" or the "Wufoos". Its a nice touch!

Maybe it mimics twitter in this behavior? What is now is the only relevant stuff. Present doesn't see beyond 5 days time horizon.

Well, actually it seems that HN search has taken the opposite approach (NOT indexing new items).

This has now been resolved! Thanks everyone involved :-)

Reply on news.ycombinator.com