Comment by tannhaeuser | Hacker News Reader

Comment by tannhaeuser | Hacker News Reader

On the other side of the spectrum, I recently did a mini review of available F/OSS options for full-text indexers /search tools. Requirements: natively compiled, one-shot/single-process per search, no long-running containers so EL, SOLR, and anything else based on mighty Lucene and Java was out of scope.

The reviewed tools were: clucene, Apache Lucy, Indri/Lemur, Xapian, and Zettair. Of those, only Xapian and Zettair could be made to return useful search results within time constraints. Sadly, I didn't get Apache Lucy to build, which along with Xapian is the only option still being maintained. clucene could be built, but delivered erratic results (it found "impeachment" but not "indictment" in the US constitution text), and was too difficult to extend into a full stand-alone app (though it apparently is being used in Informix still).

Have you seen Bleve? https://github.com/blevesearch/bleve

I have only tried it for an hour or two, and can't speak for its search result quality, but it seems promising.

On my laptop, with an index of 10,000 documents each containing 200 random words, it takes about 110ms to perform a fuzzy search. Half of that is the actual search, the other half is the process/initialization overhead.

Thanks. Will try.

FWIW we will likely open source our search engine/IR technology sometime next month (hold up is the documentation and cleaning up some parts of the codebase).

It's written in C++, and the core features are performance, simplicity(operation and extension of functionality) and a sane, simple API (the codebase is quite small as well, at about 14k lines of code).

Its similar in some regards to Lucene, but does away with most of the exotic features and instead is optimised for rich queries support, fast execution and extensibility.

It should be on GH sometime in April.

Looking forward to it. Has it a project name yet?

Yes, its called Trinity.

Sphinx search is C++, and uses little memory compared to Lucene. It has many advanced features like Lucene.

https://en.wikipedia.org/wiki/Sphinx_(search_engine)

I didn't consider Sphinx because it's using a RDBMS as store. For the same reason, I didn't check out sqlite-fts etc.

Maybe I should reconsider?

Sphinx does NOT use an RDBMS as store. It just provide a convienient interface so you can query Sphinx like you would do a SQL database, just one of several APIs to interact with Sphinx. The data store is very similar to Lucene, a folder with several files.

Reply on news.ycombinator.com