Hacker News Reader

Comment by baalimago | Hacker News Reader

I've had the same exact thought! The reason why we've moved from higher to higher level of programming language is to make it easier for humans to describe to the machine what we want it to do. That's why languages are semantically easier and easier to read js > cpp > c > assembly > machine code (subjectively, yes yes, you get the point). It makes perfect sense to believe that natural language interpreted by an LLM is the next step in this evolution.

My prediction: in 10 years we'll see LLMs generate machine code directly, just like a normal compiler. The programming language will be the context provided by the context engineer.

View on HN

I've thought of this too.

But I always end up in a scenario where, in order to make the LLM spit out consistent and as precise as possible code, we end up with a very simple and tight syntax.

For example we'll be using less and less complete human sentences, because they leave too much open to interpretation, and end up with keywords like "if", "else" and "foreach". When we eventually do end up at that utopia, the first person to present this at a conference will be hailed as a revolutionist.

Only for the LLM to have a resolve clash and, while 'hallucinating', flip a boolean check.

Normal compiler does not generate machine code directly. Normal compiler generates LLVM IR code. LLVM generates assembly listings. Assembler generates machine code. You can write compiler which outputs machine code directly, but this multi-level translation exists for a reason. IMO, LLM might be utilised to generate some Python code in the far far away future, if the issue with deterministic generation would be solved. But generating machine code does not make much sense. Today LLM uses external tools to compute sum of numbers, because they are so bad at deterministic calculations.

The core issue is that you need to be able to iterate on different parts of the application, either without altering unaffected parts or with deterministic translation. Otherwise, this AI application will be full of new bugs every change.

>if the issue with deterministic generation would be solved

This can be achieved by utilizing tests. So the SWE agent will write up a set of tests as it understands the task. These are the functional requirements, which should/could be easily inspected by the BI (biological intelligence).

Once the functional requirements have been set, the SWE agent can iterate over and over again until the tests pass. At this point it doesn't really matter what the solution code looks like or how it's written, only that the functional requirements as defined via the tests are upheld. New requirements? Additional tests.

Totally agree - I'd bet there will be a bigger emphasis on functional testing to prevent degradation of previously added features. And I'd bet the scope of tests we'll need to write will also go up. For example, I'd bet we'll need to add latency based unit tests to make sure as the LLM compiler is iterating, it doesn't make the user perceived performance worse

C and C++ UB enter the room,....

I agree that the level of abstraction will grow and LLMs will be the primary tool to write code *but* I think they will still generate code in a formal language. That formal language might be very close to a natural language, pseudo code if you will, but it will still be a formal language. That will make it much much easier to work on, collaborate on and maintain the codebase. It’s just my prediction though, I might be proven wrong shortly.

You seem to have missed this part of TFA:

> That means we no longer examine the code. Our time as engineers will be spent handling context, testing features, and iterating on them

IOW there would be no human to "work on, collaborate on and maintain the codebase" and so the premise of the article is that it might just as well emit machine code from the "source prompt", hence "LLM as compiler".

Or maybe you mean that this formal language is not for humans to handle but entirely dedicated to LLMs, for the sake of LLMs not having to reverse engineer assembly?

I think that's where the premises differ: the author seems to suggest that the assembly would be generated each time from the "source prompt"

I don't know, these all read like thought experiments built on hypothetical properties that these AI tools would somehow be bestowed upon in some future and not something grounded in any reality. IOW science fiction.

> My prediction: in 10 years we'll see LLMs generate machine code directly, just like a normal compiler. The programming language will be the context provided by the context engineer.

Sure and if you run into a crash the system is just bricked.

This sort of wishful thinking glosses over decades of hard earned deterministic behavior in computers.

> It makes perfect sense to believe that natural language interpreted by an LLM is the next step in this evolution.

Which one? Most languages are full of imprecision and change over time. So which one would be best for giving instructions to the machines?

In the scheme described in the article the main input for AI would be the tests. If we are testing code outputs (and why not) the input then must be in a programming language.

Specifications need to be unambiguous but Natural Language is often ambiguous.

The level of abstraction of programming languages has been growing, yes. However, new languages have preserved precision and predictability. I would even argue that as we went up the abstraction ladder, we have increasingly improved the precision of the semantics of our languages. LLMs don't do that at all. They completely destroy determinism as a core design. Because of that, I really don't think LLMs will be the future of programming languages.

Interesting, I'd hypothesize something slightly different, that we'll see a much more efficient language come out. Something humans don't need to read that can then get compiled to machine code super efficiently. Basically optimizing the output tokens to machine work done as much as possible

I concur. not intermediate code directly machine code, even no tests. It will take human specs internally understand them (maybe formal methods of reasoning) and keep chatting with the user asking about any gaps (you. mentioned a and c, what about b) or ask for clarification on inconsistencies (i.e. in point 16 you mentioned that and in point 50 you mentioned this, to my limited understanding doesn't this contradicts? for example if we have that based on point 16 and have that based point 50,how do you resolve it. in short will act as business analysis with no (imagined) ego or annoyance in the middle by the user. from talk to walk

But this is a waste of resources. LLM should be generated in a higher-level language and then compiled.

There you go. Then an actual compiler compiles the code into the correct low-level assembly for the actual linker to create an executable.

Congratulations. An LLM is not a 'compiler'.

Reply on news.ycombinator.com