Comment by jimmytucson | Hacker News Reader

Comment by jimmytucson | Hacker News Reader

Parent Full thread

It is just as "vibe-ish" as vector search and notably does require chunking (document chunks are fed to the indexer to build the table of contents). That said, I don't find vector search any less "vibey". While "mathematical similarity" is a structured operation, the "conversion to high-dimensional vectors" part is predicated on the encoder, which can be trained towards any objective.

    > scaling will become problematic as the doc structure approaches the context limit of the LLM doing the retrieval

IIUC, retrieval is based on traversing a tree structure, so only the root nodes have to fit in the context window. I find that kinda cool about this approach.

But yes, still "vibe retrieval".

It doesn't look like it's just root nodes from the structure, it appears to be the entire structure including a summary and excluding the text content itself:

    {json.dumps(tree_without_text, indent=2)}

The end result is that a pre-summarized digest is input in each prompt, the LLM selects whatever it decides on.

The pageIndex value add here is ostensibly the creation of that summary structure, but this too is done with LLM assistance. I've been through the code now, and what I see is essentially JSON creation and parsing during the index process that has LLM prompts as the creation engine for all of that as well.

Yes, it is technically vectorless-RAG, but it gets there completely and totally with iterative and recursive calls to an LLM on all sides.

Looking through the rest of their code & API, the API exists to do these things:

    1: Create your ToC using unsupervised[1]  LLM calls.
    2: Serve your ToC to an LLM when searching or querying your doc base
    3: Be your document store to return hits from #2

[1] Unsupervised in the ML sense, not as a value/quality judgement.

Reply on news.ycombinator.com