Pretty cool. A little slow for a search engine, have you tried in-database embeddings for semantic search like PostgresML?
Agreed it could be faster for uncached queries. The embeddings retrieval itself is actually pretty fast (uses pgvector). However, I found that having a LLM rerank results + generate summaries related to the search query made results more useful, which is what accounts for much of the latency.
Maybe I should make that a user-customizable setting!
If you'd like free hosting, feel free to reach out. I'm one of the founders at postgresml.org.