Hacker News Reader

I'd be interested to know what drove the technology choices for Java and Apache Solr.

For one, Lucene[1] is quite amazing. With a slight stretch of imagination, you can think of it as a document oriented database, with all fields indexed by default. Imagine loading 100M rows with 100 columns into an RDBMS and then index each column[2]. You don't necessary do that. With Lucene - and SOLR and elasticsearch for that matter - it's not even a thought. Additionally, you get to tweak your indexing with dozens of field-tested analyzers.

[1] http://lucene.apache.org/core/

[2] we do this (over 300G of complex data) with a tool called solrbulk and on a single server in less than four hours, https://github.com/miku/solrbulk

I agree that Lucene is a mature solution with many excellent use-cases.

For this portion of the stack why would it be preferable to start with Lucene rather than Elasticsearch? (for all of the distributed scaling benefits)

I would probably start with elasticsearch and establish a business case first. Only once that's proven and some need for lower level customizations arises, I would descend the stack.

Perhaps it's because Elastic Search shouldn't ever be a primary data store - it will eventually eat your data.

That said, looking over Chronix I can't tell if it was supposed to be a primary data store.

Reply on news.ycombinator.com