Hacker News Reader

Comment by CuriouslyC | Hacker News Reader

I regularly put 50k LoC codebases in gemini, it has a 1M context window and actually uses it well.

I've had the opposite experience. If I give it that much context it starts to hallucinate parts of the application that it very much has access to look up. This only starts happening at large context windows.

Depends on what you're doing. Too much context and code generation gets sloppy, but it does a decent job attending to the large context to answer questions, analyze control flow, investigate bugs, review style consistency and guideline violations, etc.

Let me know when it handles 1.5M lines.

Let me know when you meet a person that can handle 1.5M lines, because most people I've worked with can't. Certainly I've never worked on something that required even reading that much... Especially when targeted search options exist to find just specific function/class implementations as needed.

I have a large team that works in a 1.5M line codebase. Not everyone is familiar with every line, but we regularly make changes that would blow through the usable context window of LLMs (which is, in fact, much smaller than advertised).

Is a messy 1.5M lines of tightly coupled code best practice? Of course not. But it evolved over about 20 years and processes tens of billions of dollars of financial transactions. In my experience, it is archetypical of real-world software for a large successful company.

I use LLMs where I can and they're incredibly useful. But their limits are severe compared to a good human software developer and the shortcomings mostly revolve around their tiny context. Human neuroplasticity is still champion.

> 1.5m line codebase

I work regularly with AOSP code (~3mn). While LLMs (Copilot w/ Claude Sonnet 3.7, in my case) cannot gobble all of it up, they have no trouble answering my queries, for the most part, from submodules. If nothing else, using LLMs has dramatically reduced the time it takes to understand a new code file / submodule / a range of commits.

I'm going to wildly guess that AOSP is part of every LLMs training set, so that feels like cheating somehow :-)

Sure, they work in that huge codebase, but for any specific task, the number of lines your team keeps in their short/long term memory is surely much much smaller than that.

he will probably get back to you in a year then

Reply on news.ycombinator.com