Hacker News Reader

emil_sorensenon Retrieval's bottleneck is moving from finding to selecting

OP here! Happy to jump in to answer any questions. :)

emil_sorensenon Benchmarking retrieval for agents on messy real-world company knowledge

OP/founder of kapa (YCS23) here. Happy to answer any questions :)

emil_sorensenon How we index images for RAG

happy it helped!

emil_sorensenon How we index images for RAG

it's a cost/latency trade-off in production + very use-case dependent

emil_sorensenon How we index images for RAG

Thanks! Yep fixed

emil_sorensenon Is the doc bot docs, or not?

Accuracy drops hard with context length still. Especially in more technical domains. Plus latency and cost.

emil_sorensenon Is the doc bot docs, or not?

Docs bots like these are deceptively hard to get right in production. Retrieval is super sensitive to how you chunk/parse documentation and how you end up structuring documentation in the first place (see frontpage post from a few weeks ago: https://news.ycombinator.com/item?id=44311217).

You want grounded RAG systems like Shopify's here to rely strongly on the underlying documents, but also still sprinkle a bit of the magic of the latent LLM knowledge too. The only way to get that balance right is evals. Lots of them. It gets even harder when you are dealing with GraphQL schema like Shopify has since most models struggle with that syntax moreso than REST APIs.

FYI I'm biased: Founder of kapa.ai here (we build docs AI assistants for +200 companies incl. Sentry, Grafana, Docker, the largest Apache projects etc).

emil_sorensenon Writing documentation for AI: best practices

Thanks for the feedback. We should definitely add that. :)

emil_sorensenon Writing documentation for AI: best practices

OP here. It's kind of ironic that making the docs AI-friendly essentially just ends up being what good documentation is in the first place (explicit context and hierarchy, self-contained sections, precise error messages).

emil_sorensenon Evaluating modular RAG with reasoning models

We focus mainly on external use cases (e.g., helping companies like Docker and Monday.com deploy customer facing "Ask AI" assistants) so we don't run into much of that given all data is public.

For internal use cases that require user level permissions that's a freaking rabbit role. I recently heard someone describe Glean as a "permissions company" more so than a search company for that reason. :)

emil_sorensenon Evaluating modular RAG with reasoning models

Super cool! Yep, a lot seems to get lost through distillation.

emil_sorensenon Evaluating modular RAG with reasoning models

I suspect you're right here! Excited to get our hands on the non-distilled o3. :)

emil_sorensenon Evaluating modular RAG with reasoning models

Yep even with a small bump in performance (which we only saw for a subset of coding questions), it wouldn't be worth the huge latency penalty. Though that will surely go down over time.

emil_sorensenon Evaluating modular RAG with reasoning models

Curious if anyone else has run similar experiments?

emil_sorensenon AI hallucinations: Why LLMs make things up (and how to fix it)

That's a great point. Reminds me of the "feature, not a bug" Karpathy tweet [0].

[0]: https://x.com/karpathy/status/1733299213503787018?lang=en

emil_sorensenon AI hallucinations: Why LLMs make things up (and how to fix it)

I find it so interesting that it's possible to develop a "feeling" of a new model.

emil_sorensenon Optimizing technical docs for LLMs

Extensive LLM evals. It's a real rabbit hole.

emil_sorensenon Optimizing technical docs for LLMs

They don't have to be rendered in the browser, but having all of the structures and section headings help humans to, so I would recommend it. :)

emil_sorensenon Optimizing technical docs for LLMs

Yes, definitely!

emil_sorensenon Optimizing technical docs for LLMs

Fair point. Although good writing for humans = good writing for LLMs and vice versa. So I'm hopeful this new excitement around AI for docs if anything will just encourage folks to put even more effort into writing great docs.

emil_sorensenon Optimizing technical docs for LLMs

You nailed it. All of above sources are also super helpful for LLMs, but you have to be careful about how to ingest/parse them. For example for a Discourse Forum only including questions that have been marked "Resolved" by an official team member can work quite well. Same goes for Discord/Slack forums and GitHub Discussions etc.

emil_sorensenon Optimizing technical docs for LLMs

Exactly! It's very much garbage-in-garbage-out.

One surprising (to me at least) benefit of hooking up an LLM to your docs is that it is actually a really useful way to find gaps in your docs. For example, when an LLM cannot answer a user question, there's a good chance it's because the answer is not documented anywhere.

emil_sorensenon Optimizing technical docs for LLMs

Author here. Happy to answer any questions!

emil_sorensenon Launch HN: Greptile (YC W24) - RAG on codebases that actually works

Yep, agreed. As a founder of another YC company that's done a bunch of RAG on code and docs, the accuracy boost you get from having good docs makes a massive difference. You really do need a human interpreter to show you "how to use" a product.

emil_sorensenon Launch HN: Encord (YC W21) – Unit testing for computer vision models

This looks promising - but how is this different from tools like Aquarium Learning or Voxel51?