Yes, I use "scrapy crawl <spider> -o <spider>.json" (see bootstrap.sh). Sphinx can not index JSON files directly, but it can index an XML stream written to stdout by a given command. So I have written a script (sphinx/xmlpipe.py), that reads the JSON files generated by scrapy and writes the crawled recipes in sphinx's XML format to stdout.

There are alternatives to Sphinx? ;)

* Sphinx is ridiculous fast, as you can see when searching. But even building the index takes only 340ms (from which 220ms are spend by the python script that generates the XML) for 1699 recipes on my 3 years old notebook.

* Sphinx don't require a RDBMS to index documents from. It can index documents from any source. You just need to write a simple script that brings the documents in the XML format expected by sphinx.

* Sphinx is not only a full text search engine. It is also a multi-value store. You can add extra information like the title and url to indexed documents. And so you don't need an additional database.

* I need the ability to limit a fulltext search to sentence boundaries. I don't know if there are other fulltext search engines that can do that.