Hacker News Reader

Comment by mrweasel | Hacker News Reader

Apparently it's not obvious to everyone, but if you can't write code, you can't review it. I do know people, and companies, that says: "So what, we ask Claude to write the code, Codex will then do the review". The thing that then strikes me as odd is that they still ask for the code in Python, Java, or some other high level language.... Why? Just ask Claude to dump out assembly, or a compiled binary, but no, they don't trust the LLM that much. They still want to be able to read the code. So they need developers that can read, debug and reason about the code, yet they don't want to give them the training that's required to do this?

View on HN

They don't have Claude write assembly because there is no training corpus on people making CRUD apps in assembly.

I'm as hateful of LLMs hollowing out the job market as the next guy, but the reality is the frontier LLMs are really good at writing anything that's been done and documented on the Internet a million times and unfortunately most of what software devs have been doing the last couple decades is shitting out cookie cutter CRUD apps.

I have my doubts about whether the state of the industry is going to advance as long as we're having LLMs do all the creation, but that's another diatribe.

Claude is perfectly capable of writing assembly. Here's a working (basic) Prolog interpreter that Claude Fable 5 wrote in WebAssembly in 61 minutes for $16.75 in token costs: https://github.com/emk/fable-wasm-prolog/blob/main/prolog.wa...

WebAssembly is slightly easier than real assembly, but here Fable used WASM GC extensions, which are poorly documented and not yet super common.

Fable didn't even need to debug it; I believe essentially all the assembly worked correctly on the first try.

I have feelings about this, but I'm not pretending it isn't real.

I have a personal game framework that I have LLMs write games in, which is in AssemblyScript. AssemblyScript is certainly closer to TypeScript than it is to WebAssembly, but it's still this thing where the host shares some big chunk of memory with the script and you pick some memory locations to read and write as your means of exposing APIs, and there's not a lot of training data on games written in AssemblyScript and even less in my game framework specifically (none) - and the LLM does an excellent job.

I guess since fable has learnt deeply about 10+(maybe 100+) popular programming languages, it can just easily reason about the new 101st programming language.

This is for games that run purely in the browser?

No, I typically want to target either mobile or PC first - but I strongly want stuff to run well in the browser too, for early playtesting and for demos at the very least.

WebAssembly has as much to do with assembly as JavaScript has with Java. I highly doubt the "slightly easier" and my attempts with LLM's and assembly so far were largely disappointing, but I also think there is no compelling reason this has to be the case.

I have written assembly for about 5 different processors, including 65C02s, 680x0s, cute little DSP3210s that managed the CPU cache manually, utterly cursed TI320C40s, and (of course) a bit of Intel. WebAssembly is simpler than some of these architectures in some ways.

But it's not that much simpler. And once you add the WASM GC stuff, WebAssembly gets weird. It's a Harvard architecture with separate value memory, linear memory, GC memory and "tables", all accessible in completely different ways, with a weird mandatory type system (especially for the GC stuff). And the docs are often terrible. And yes, I've also written WebAssembly by hand.

So yes, I would, overall, classify WebAssembly as "slightly easier". But not dramatically so. And the training data for actually writing non-trivial things by hand isn't that great, not compared to something like Intel assembly.

(Don't talk to me about TI320C40 assembly. If Fable can one-shot a Prolog interpreter written using that without finding a reference manual, it's time to hang up my hat and learn to make goat cheese.)

The guys with unlimited Fable/Mythos access are for some reason incapable of producing a flawless Claude Code app built entirely in native assemblies.

You’re getting down voted, but it would really be nice if they rebuilt the thing in Rust or something like that. I’m just tired of these enormous JavaScript bundles pretending to be desktop apps. Especially when we need to squeeze as much juice as possible out of our machines.

They're so busy winning that they don't care about improving their own web apps ;)

they use React to render their TUI :)

There’s a very real dichotomy at play here. Position 0: humans depend upon AI. Position 1: AIs depend upon humans.

At first blush, it seems achingly obvious that position 1 is true, whilst position 0 is a false play by con artists.

Well, at first blush I agree! But first blushes are notorious for being famous last words (blushes).

You see, we always knew that the tool shapes the hand. That is, as we use computational (discrete) devices, we ourselves become more computational, discrete.

But what we did not anticipate is that the tool would fool others as the ACTUAL HAND. I am so fooled. Daily.

My friends. Does technique encompass being? Answer me this!

Just interesting to see Claude’s hourly rate is around $17.

Much cheaper than a human software engineer. But comparable to the wages for some human workers.

As soon as human software engineers are Ubered away from the field, the price will ramp up to that of a human software engineer or more, have no doubt.

ROI won't make itself up.

It is hard to believe that no one sees this coming. I can’t imagine it will happen any other way.

Let's hope open models will continue to improve and avoid the price ramping.

Then everyone just starts running open weight models locally.

Running it on RAM and GPUs which cost like a private jet plane, or have to be smuggled in like cocaine due to import/export tariffs or bans.

U.S. is actively considering restricting highly advanced "open-weights" models due to cybersecurity and national security concern.

Chinese models will be enlisted in NATS - the National Terrorism Advisory System.

Can't have the fat, rich big boys who give the campaign contributions fail.

claude helped me build a functional visual modem (using the cameras and microphones/speakers) for mac/iPhone in about a day and <100 prompts. i’ve worked on mobile apps once 10 years ago and have never built one from scratch. i've built a handful of UIs in 20 years on different platforms. my actual work was mostly in systems software. so it would have probably taken me weeks to months to build this from scratch. it would have cost substantial money to outsource this and iterate on ui and core algo as i did.

i've been curious about the feasibility of this concept for quite some time and now i have an actual example to study and measure and hack on, for a fraction of the cost of my pro sub. it wasn’t trivial, i doubt a non-swe could get it done in that time, but i'm a pretty happy customer overall.

That’s around 28K EUR/year. I know some junior-mid engineers that earn that (and they are good) in southern Europe.

But now human will produce the same amount of work in an hour - not even close...

I would assume a human would require multiple hours for the same output though

And ofc the Chinese workers are a lot cheaper.

> Fable didn't even need to debug it; I believe essentially all the assembly worked correctly on the first try.

so you don't know if it works properly or not

Fable wrote a pretty decent test suite covering typical Prolog programs, including things with non-trivial execution patterns like "append" that do complex backtracking on multiple branches. And I've run a modest number of test queries by hand, trying a few things. It's entirely possible that there's a bug there somewhere. But it's better than I would have done on the first try, implementing a Prolog interpreter in assembly. And I've worked on actual production compilers a few times.

I am honestly not happy about the way that models can now just take what should have be a fun multi-weekend project and knock out in a couple of hours. But I'm not going to pretend that Fable is stupid, or that it did a bad job on any of the test projects I gave it. It struggles more on big, messy real-world code bases, absolutely.

It's also conveniently forgetting that AI was trained on the whole internet, a corpus of code including every garbage solution in SO wrong answers or random junior blogs. I didn't see any claim AI could select "best practices" from their sources, so putting AI to correct another AI has high chances of statistically selecting the same wrong solution, because that was just a common thread in those old posts they used as reference.

I think you're pretty wrong here, but I'm no expert so anyone who wants to correct me please do so.

As I understand it, there's post-training step that does exactly that, they get real developers to select good and bad code and help the AI figure that out. AFAIK it's the reason why Claude was able to leap-frog OpenAI in coding last year, they did this refinement step a lot better.

And now they've taught the older models this, they can use existing models to bootstrap the newer models without humans involved, and bring humans in to just assess the trickier stuff.

Claude's coding efficiency is largely due to their control plane governing the actual execution of the model - rules, evaluators and basically looping over results. Not related to the model training or fine tuning.

The models were already much better before Claude Code came out, let's not rewrite history here.

Yeah I had it remake my favorite TI-89 graphing calculator game in Python and it one shotted it perfectly.

Which effort setting did you run this with?

> They don't have Claude write assembly because there is no training corpus on people making CRUD apps in assembly.

I suspect that despite its translation abilities, this is true, but I'd like to see it do things in languages that are more or less appropriate for tasks to see how much the training corpus matters vs. its ability to translate. Assembly is a bit of an extreme example because you're either writing it as close to C as possible (C is essentially portable assembly) or you're writing complex, unreviewable code that happens to work. And who know if it's been trained on register allocation, or resorts to doing everything on the stack because it works.

> C is essentially portable assembly

Not really, assembly shows you what the machine instructions actually are. In C you have loops, functions structs. All things that make no sense in assembly.

Both points are specious. C disassembly is usually not an orthogonal translation of the structures in the code, usually due to optimisation.

> In C you have loops, functions structs. All things that make no sense in assembly.

One can reasonably infer that a loop will roughly translate to some type of conditional jump instruction. Function structs are just pointers and call instructions. The former point was that there is little runtime getting in the way of how the constructs could be expressed, and a long time ago it may have been the truth, but not for modern compilers.

Loops (conditional JMPS - JE/JNE/JZ/JNZ/JB/JNB.. and a dozen others) and functions (CALL, RET) exist in assembly. Goto is there too (all JMPs). C makes assembly sense.

Oh that's an interesting thought. Goto's are considered harmful, because holy hell as a human trying to understand and reason about code, the spaghettifcation if someone else used gotos all over the place and I have to reconstruct their thought process just doesn't sound like fun. But if we're no longer writing code by hand (which, I gotta be honest, I'm using AI to write a lot of code for me these days), then does that still matter? If a human isn't going to have to understand the code, can we get more performance out of existing systems by considering goto's useful?

goto is already useful as an end-of-function cleanup. You pop one at the end of each function (the naming is local to the function) and that's the place where you free ram if the pointer's not null and close the file if the handle is not null and so on.

Then you don't have to litter all of your if/else clauses with hundreds of memory frees and cleanups - you just "goto"

The fact that this is controversial to some people just shows how rules like "Dont use goto" become stupid - if one is trying to make code simpler then there are cases were not using it is the more confusing and error prone choice.

As for "can we now do anything" well if you don't ever need to debug "anything" then yes and if you have any small doubts about your LLM then no.

I assume there's a clear technical reason why GOTO is harmful, it adds to the complexity of the code, makes it harder to understand and fix and evolve. Wouldn't the same difficulties affect the LLMs ability to understand and fix and evolve the code as well?

As a human with a human level of working memory, the number of things I can keep in my head and map to concepts and functions is unfortunately limited. An AI running on an Nvidia supercomputer has higher limitations on what it can hold in its working memory, so could theoretically handle more than my human brain can.

But it won't because it's a language model no an artificial intelligence.

Compilers only really use them for things like break and the end if blocks. There isn't much to gain by being able to execute anything arbitrarily, and then go...where? There might be places here and there within a function where compilers already take advantage of it, but I doubt it's useful in the way you think.

in C it's quite normal to set a return code and jump to a cleanup section where all the buffers are freed, all the files closed and so on.

LLMs are perfectly capable of translating between programming languages at this point. The main bottleneck for them is logic, not language.

And the amount of logic needed to understand code, including how to fix it or modify it further depends on how complicated the code-structure is.

I wonder about this: Does LLM write better code in a better language?

Humans often have more difficulty fixing existing code than reqriting it from scratch, and un-structured un-constrained code would be a big reason for that.

But maybe it doesn't apply to an LLM if it always rewrites all code from scratch?

> I wonder about this: Does LLM write better code in a better language?

The whole point is to rewrite everything in a language you don't understand and then ignore whether it's better because you can't tell.

That language, of course, must be Rust.

What's there to advance to?

Without a revolutionary new platform to build apps on that no one has ever developed for before, there is basically no reason to believe there is any software left that has some business or economic value that hasn't already been written.

This gives "let's close the patent office"

> This gives "let's close the patent office"

This is a really good idea for completely different reasons. :-)

This is probably true in the consumer facing software space. But in business, industry, and academia, often there are novel data generated by some new or refined process. And this needs new software that correctly handles both the nature of these new data and meet the goals the data is collected to advance. This sort of software is also usually poorly represented in training sets, if represented at all.

If you think we're "done", you have no imagination.

What are you gonna make? Yet another CRUD app? An API subscription? A game? A mobile app?

We’ve created software for virtually every place we can put software. There’s nothing new.

It’s like bridges. We’ve seen all the ways bridges can be built by now. There’s nothing new left to discover.

> There’s nothing new left to discover.

It's the end of history. What could we possibly discover about a series of technologies that are already nearly 100 years old now?

I've heard people say that various things are "solved" now because of LLMs too -- programming included. This implies we've "solved" thinking. I'm worried about these sentiments.

https://en.wikipedia.org/wiki/End_of_history

When Gutenberg invented the printing press, he could not have predicted the teleprompter. Now that code is easy to generate, it’s plausible that we will use much, much more of it than before.

It’s also a bit ironic to imagine that we’re at the end of new software ideas on a site owned(?) by YC.

Are teleprompters a net positive achievement? They were needed for live television, they will be superseded I guess by AI generated news avatars. And 24-hour news coverage is in the business of generating content non-stop, to sell advertisements, regardless if there are newsworthy events or not, the machine never stops producing.

it's also plausible that with an ability to generate as much code as we want with little effort, we will quickly discover how much code we actually need, and then plateau at that point.

Personally, I think we've already reached that point.

Even at YC, I have not really seen any startup doing anything interesting where their main value prop is enabled by new proprietary code that does stuff no one else can already do.

We have more code than we need. Proof for me is every "smart" device that worked better before we put a computer in it.

Exactly, this is the true problem with the world today: too much code.

Mature tech evolves more slowly. Think about cars, there's nothing new about them any more. But wait, what about electic cars? Aren't they a new thing? And driverless cars. And soon flying taxis. Automobiles are an old invention but they are still evolving.

The reason the evolution of cars has slowed down so much is mostly regulatory compliance.

With the current ongoing revolution in home manufacturing (starting with 3D printing; currently there is a lot of work done in home CNC machining), it would be easily possible to get a huge leap towards "print your own car".

The reason why you see nothing of this is basically red tape.

Why did we even need bridges in the first place

Well, a short time ago somebody made the first LLM. Was that the end of it?

this implies that the only thing that can change what needs doing are technology platforms, which about 6000 years of human civilization would indicate is not the case.

that you don't see this as inevitable worries me dearly

couldn’t the frontier labs simply compile their existing source code training data and have an assembly corpus to train on?

It's still possible to make CRUD apps in assembly with an AI agent but it would be a research project.

By "research project" you you mean by people who understand assembly? Because then we're back to where we started.

I mean for the agent. They are good enough to conduct research now.

LLMs are designed to imitate humans not compilers. Hardly anyone is capable of writing modern optimized Assembly code by hand outside of a few snippets here and there (not in a reasonable amount of time at least)

>They don't have Claude write assembly because there is no training corpus on people making CRUD apps in assembly.

I'm disputing this. You can have a training corpus in assembly as big as any other language: just feed the compiled result(in assembly) of the CRUD apps to the LLMs.

> Apparently it's not obvious to everyone, but if you can't write code, you can't review it.

There's more to it than that: writing is thinking. If you stop writing code, you aren't thinking anymore.

Many argue that they're now thinking at a higher level (maybe they weren't before?), but, guess what, that high-level design can be done better by the LLM than by you anyway. It's only temporary.

>> There's more to it than that: writing is thinking. If you stop writing code, you aren't thinking anymore.

Humans have been thinking long before writing was invented. Why is code special?

> Humans have been thinking long before writing was invented.

And look how poorly they did it until they invented writing.

> Why is code special?

You don't get better at thinking without doing. With code the "doing" part is writing.

In your view, it is reasonable to expect HS students to thoroughly read a book on trigonometry and then ace the exam? We know, from experience, that only by doing problems does the student actually learn. We also know that when you stop doing problems the facility atrophys.

Why is code different?

> And look how poorly they did it until they invented writing.

How can you know how poorly they did it?

> How can you know how poorly they did it?

Their lack of progress. Humans existed in the same basic form for around 100k years before they finally invented agriculture, which led to some improvements, and then rapid progress once writing was invented.

Improvements or just new ways to be occupied and more work? It seems life without agriculture and the like was way simpler and chilled.

Until winter.

Humans have been storing food for winter for hundreds of thousands of years before agriculture and the invention of writing, you think they all died off or something? People whose crops failed instead were more heavily affected in winter than hunter gatherers.

Of course, but the surpluses made possible by agriculture meant occasional starvation became a much rarer event

No, people starved way more with farming than with hunting and gathering. All in all you're under very severe misconceptions as to how farmers and hunter gatherers lived which is not supported by the evidence.

https://pmc.ncbi.nlm.nih.gov/articles/PMC3917328/

That study controls only for habitat quality, which I feel is not really relevant: hunter-gatherers are, by definition alone, nomadic. They move to where the food is!

IOW, hunter-gatherers follow the best habitat, so controlling for habitat is always going to have the conclusion this paper reached.

I'd be very interested in other studies that support this conclusion; after all, that's how science works - a single study proves nothing if many studies conclude the opposite.

> then rapid progress once writing was invented.

While I mostly agree with this notion, given that all of our understanding of human progress historically comes from writing, there's a selection effect happening here.

Like, why do we talk about the Dark Ages? Because there was less writing, so obviously things were terrible.

> While I mostly agree with this notion, given that all of our understanding of human progress historically comes from writing, there's a selection effect happening here.

There's a bit of a selection bias there, because writing itself is used as a measurement of progress.

But, ignoring writing as a measurement of progress, we can pretty much see that for about 100k years the only tech were primitive tools.

Even after the domestication of wolves (about 40k years ago) there were no visible improvement in tech. After agriculture (10k-12k years ago, maybe?), there's visible progress in tech (tools), but not by much, and visible improvement in societies (also, not by much).

Enter writing - those civilisations that had more advanced writing progressed rapidly, and those that did not barely progressed at all, tech-wise.

Because not all civilisations progressed at the same rate, we can see quite clearly that writing is highly correlated[1] with progress.

---------------------

[1] While correlation does not imply causation, as the data mounts it lends more and more weight to the causation bit.

If writing is a natural tech you unlock on an already exponential curve it would always correlate with progress. And so many thousands of other tech.

> If writing is a natural tech you unlock on an already exponential curve it would always correlate with progress.

That's a big "If". The observation we have is that civilisations that never invented writing simply died out with no tech-tree advancement; and this happened for 100k years at least.

We've got 100k years of "no writing, no tech". We've got around 5k years of "with writing came tech".

The actual tech to write was there for much of the 100k years (cave art), but without writing, the tech never improved.

>> And look how poorly they did it until they invented writing.

> How can you know how poorly they did it?

You could probably get some sense by comparing illiterate people to literate people.

>>Humans have been thinking long before writing was invented.

>And look how poorly they did it until they invented writing.

Yeah, look how poorly they did it until they invented writing.

[peers into the dim ancient past]

Sorry, my time-machine spectacles are on leave or on strike today.

And congrats on being a multi-millen(ium)arian, and a mind reader, who could both 1) exist so far back in the past, and 2) be able to know the thoughts of people who didn't write them down, because they didn't know writing.

It's like math in school. I wasn't good at math in school because I thought that understanding the concepts was enough. But without actually putting pen to paper and drilling through math problems you quickly get lost. It's a skill issue.

I specifically liked math and physics because these were something I didn't have to memorize or repeat constantly. While other subjects felt forced rote memorization. Same with code though. Writing things down feels like a chore mostly, so I am personally glad AI is here.

I didn't mean rote memorization, which I agree is what other subjects were like, we used to say "campaign learning", because we studied the material just for the exams. But with math problems for me the trouble is that I didn't put in the work, I would see for example equations solved on the blackboard and it made perfect sense at the time, but because I would skip practicing at home, I would forget some of the steps and would get stuck.

> I specifically liked math and physics because these were something I didn't have to memorize or repeat constantly. While other subjects felt forced rote memorization. Same with code though. Writing things down feels like a chore mostly, so I am personally glad AI is here.

I am skeptical that you were able to get any good at Maths or Physics without drilling problems.

have you tried thinking for a long time without writing anything down?

You will get to a point that you have to write it down so you don't forget but more importantly check if it even compiles.

Yes in three specific environments for varying durations. Lying down to sleep, showering and driving.

Each last for a different amount of time and you generally can’t write or even read anything. But in my personal experience, all three lead to solving the most difficult software problems.

Fair. But in all 3 did you very quickly write down the solution as soon as you were able?

My best thinking is when I am going for very long runs, when I write down the thought, my brain stops thinking about it, considers it done and moves on.

When writing, thoughts become forced and linear in my view as opposed to floating in a specific way from point to point. Like I lose the natural creativity.

Writing is about remembering or communicating, so it is summarizing the thoughts, but it is not thinking itself, it is more so thinking about how to communicate efficiently or how to make a note so you don't forget. It is not creativity for me.

Kind of imagine building an idea in your head while running and then you write down the summary of it.

I see, I'm the same too. Throughout my day my thoughts would be flying and zooming around. Some of them good, most of them not good or otherwise useless.

When I sit down to try and write them down, I blank out. Or the thoughts that I thought were good/useful, I second guess when writing them down. Perhaps it's just lack of practice on writing them down (I'm trying to use a typewriter to do so recently).

writing is a way to get your brain to put thoughts into a form that someone else will be able to reach the same understanding as yourself. The effort to explain oneself often leads to better understanding: Duck Debugging basically.

Writing changed everything. You barely (if at all) can tell me what humans in the past were thinking without relying on some written artifact.

That is a joke, right? E.g. as in there was no writing therefore no evidence of people having had thoughts?

I think writing is summarizing your thoughts outside of thinking. Thinking happens at least for me before I write. I know what I want to do before I write, and writing is just one medium to get it out there. If there was a faster way, to get my thoughts there I would use that.

> Thinking happens at least for me before I write. I know what I want to do before I write, and writing is just one medium to get it out there. If there was a faster way, to get my thoughts there I would use that.

A lot of people observe that their thoughts get better when they do the work to write them out. It's not just a straightforward encoding process (as maybe a software engineer would conceive of it).

Yes, it gets better, but not because of writing, it gets better because of building, playing it out and observing the results. The trial and error. It wouldn't matter whether I built via voice to command AI or wrote the code myself.

E.g. I iterate and see the architecture or the system work out, or I get other ideas because of something I don't like, etc.

> Yes, it gets better, but not because of writing, it gets better because of building, playing it out and observing the results. The trial and error. It wouldn't matter whether I built via voice to command AI or wrote the code myself.

This is a poor argument. The trial and error is the LLMs trial and error of your intention (ironically driven by you), not your trial and error.

It's like saying "It wouldn't matter if I instructed an LLM to solve my trigonometry problem or if I wrote the the solution myself; either way I'm learning trig".

> E.g. I iterate and see the architecture or the system work out, or I get other ideas because of something I don't like, etc.

What exactly are you needed for in this scenario, other than saying "make me $PRODUCT"? The LLm is going to be a lot better at architecture than you are. Your role is limited to A/B testing the result for human use.

You’re not wrong, but there’s a nuance here. More often than not our thoughts fool us into thinking we have something clear when we have only a vague idea of what you want to do.

I kinda [wrote about this before](https://dielsonsales.github.io/2023/05/11/starting-a-new-blo...) but the takeaway is that writing forces you to structure your thoughts, giving them a final shape and enables you to notice flaws or gaps in your own thinking process. It’s impossible to do this without writing.

It would be good to have specific situations here I guess, otherwise it's quite abstract, and I'm not sure what to make of it.

As to your post, I think rather than writing it's the act of pressuring yourself to figure the thing out that is making you more knowledgeable about the thing.

You are putting it out publicly out there so you really want to be certain that you are correct, and therefore you do a lot of research.

If you wanted to be accurate and prove a point, the motivation mechanism could vary, but also the eventual communication mechanism could vary. It doesn't necessarily have to be in writing, it could be a live debate or simply something else you built. The writing or the product are end product, but the process of wanting to produce something is what made you research and to understand.

You could be writing to yourself into a Google Docs, and not care and you would not go deep to research since there's no incentive to be accurate.

E.g. I could be doing a home project whether hardware, software or DIY, that in order to produce this output I have to learn and think. Similar to as good set of words to communicate some idea. But also I could build some slop without thinking and I could produce slop writing without thinking, and about nothing special.

I’m finding it all boils down to cognitive differences.

Some people find code easier to read than the English description. It’s more precise and many experienced devs can scan it and know what’s happening

Many other people can’t read code. Or they find English easier to read than code.

Thats not a knock on anyone. Maybe the latter will rule the world because the former focuses too much on irrelevant details. Or maybe me there are just different types of problems that need differing levels of attention to detail.

Sure, read the English description.

How can you be sure that's what the code the LLM wrote actually does?

The code tells you HOW, the English should be telling you WHY. Both are usually needed IMO. Even a tiny bit of "why" can be extremely helpful to understanding in my experience. I don't want generated AI garbage that merely describes the code - I want the reason the whole thing exists and what was wanted by the person who wrote it.

AKA "Direction", which obviously the LLMs follows, and suck at coming up with by themselves. Humans though, when they have purposeful direction and manages to execute it, ends up creating a (small) beautiful life-changing revolution almost.

Some heart surgeons might only be able to explain to the nurses their ideas on how to carry out the surgery, but are they still heart surgeons?

I wonder how LLMs could be compared to compilers. I personally think the nondeterministic behavior of LLMs are the biggest distinguisher.

Pretty sure Casey Muratori and Demetri Spanos cover why in one of their videos online.

Less about “trusting” the llm and more about how complex it is to work with binaries due to machine code being different per machine and hard to interpret the context of the code as well as offsets.

In that sense because high level languages come with the ability to add context to what code does. It’s like the understanding a human has when given decompiled C code ghidra gives you vs C source code a developer wrote.

Also the compiler helps the llm write “compiled / working code”, if it just spat out machine code it most likely not even run at all.

But yea generally if you can’t write code at all, reviewing it is even harder.

First of all, obviously there's more training data on high level languages than assembly / compiled binaries. Secondly, Python, Java or even C is portable to another device. You know, one of the main points of high level languages: that you don't have to rewrite your app from scratch each time you want to change the machine it runs on.

Thirdly, compilers also do type checking and other static analysis and dynamic checks (array length etc). So it's not only got more guardrails against "mistakes" by the LLM, but it's also most likely "easier" for the LLM to "reason" about (in the sense that LLMs can reason).

There's no intrinsic value to having humans read the code, so as soon as LLMs are good enough to work on their own, companies will no longer need to keep developers who can read code as a backup solution.

I have a few personal projects, i let codex do all the code - i do the thinking and testing.

One time, something didn't work as expected - its the first time it happened with this project. I read through the section of code and it was perfectly readable and well-written.

Turned out a plugin wasn't effecting the audio, so i just got it to pad some blank audio onto the beginning before processing it, then remove it at the end of the process. That fixed the issue, there was nothing wrong with the code but my ability to think laterally is what made it work.

We're getting to the stage where you can just ask them to write code and they will do what you want, and it writes good code. Its up to you to test everything beyond the internal tests it writes.

> Just ask Claude to dump out assembly, or a compiled binary, but no, they don't trust the LLM that much

No, people do advocate for this. It may simply not be as portable however as something that is high level able to be compiled to assembly for many machines.

It's the logical outcome of believing that human skills aren't important and that you can "think at a higher level". You can't spend all your time thinking at a higher level if you cannot fully trust the output and the assembler argument is just taking the thing to an extreme to make that point.

> Apparently it's not obvious to everyone, but if you can't write code, you can't review it.

Why review it at all? The trend is to vibe code everything with agent harnesses and tokenmaxxxing.

No defense for not writing or reading code if want to call yourself a software engineer or programmer.

However I do think there is reason to use Java or Python (as much as I loathe both) they have GC, and it'd be a lot easier for AI to fuck up memory safety in something like assembly or C.

I concur.

One has to program computers to be a computer programmer. If one just prompts LLMs, they are a software prompter. I think both can fit under the title of a software developer or perhaps even software engineer.

This isn’t obvious at all. Editors can evaluate and improve the work of writers even without being bestselling novelists themselves. Richard Williams coached his girls to world championships without winning one himself. Roger Ebert never even made a movie.

No one gets hurt or dies when roger ebert whiffs on a movie review. Now imagine I tell you I've built a bridge. I have no idea how to perform the necessary calculations. I have no idea how to verify the necessary calculations. But, I've built you a bridge. It kind of looks like a bridge. Will you drive across it?

Not all software is mission critical. But some software is.

Ask the ai what the game theory is for training employees... the Nash equilibrium is under training and self funded certification/training for devs in low trust environments.

dump out assembly?

aargh! haalp!

Which rock are you living under these days.

Nowadays everything is IR or MLIR ... or AIR ...

whoops, i may have skipped a few generations. hard to keep up with the dynamic pace these days. got a job to save. sorry suh.

mumble javascript mumble framework of the half-week ...

walks off muttering

No need for IRs if LLMs are all powerful. Just tell them to generate x86 and ARM and let them get on with it.

Niiice ...

I'll byte the bullet.

Hang on a bit while I register that in my memory.

>generate x86 and ARM

So that hardly anyone can review the machine language output, I guess.

HN users' minds be like:

87.654321 %: JS? yeah. mainhipsterstream. gotta be a part. heart.

smaller %: Rust? wannabe. donno how, u c.

much smaller %: C? oh no ... numeric underflow.

Congrats on moving over to the light side.

If nobody can review it that's fine since we'll be generating at least 50k LOC per day and nobody could review that anyhow. We can just get every other LLM model to review the output. What can go wrong?

token cost, for one.

bobby tables, for another.

https://imgs.xkcd.com/comics/exploits_of_a_mom.png

churning out assembly code would cost far more in tokens, wouldn't make financial sense for the improved performance which doens't matter that much for most use cases.

just to add to your excellent observation

by writing code by hand - you're the author & editor at the same time.

if you're skilled enough - you quickly get to the desired state vs the llm which while might produce a lot of code - but it likely won't give the end state you desire.

an analogy - Jason Fried gave - producing software by llm's is like bragging you're a photographer by pressing the shutter button on an automatic camera.

while the art of photography - is about producing the single perfect photo - that communicates the photographers intent (what they want the world to see & experience).

likewise in software - what we make should be deliberate.

Just so you know, most people nowadays can create decent pictures or even movies with smartphone. Not automatic camera, not polaroid, not Polaroid camera, not analog Canon FD, not cardbox camera.

Where will you put a line between photography and pressing button?

My point being - tools change with time.

> but no, they don't trust the LLM that much

Yet...

>if you can't write code, you can't review it.

I don't write code anymore and I doubt I ever will ever again.

On the flipside I review exponentially more code than ever before.

>Just ask Claude to dump out assembly, or a compiled binary, but no, they don't trust the LLM that much

It's not "not trusting" the llm its that the llm has been undergoing reinforcement learning is on coding. Plus generating assembly is extremely token inefficient.

The difference is that you already learned how to write code. New devs that go straight to genai are missing that.

Perhaps new devs should write all their code by hand until they feel comfortable doing that. Then, they can start using LLMs gradually, just a little bit at first, all the way to fully AI generated code.

However, that would just make the difference in productivity between junior and senior developers even more pronounced! It’s really difficult to see how things will play out in the future without junior developers disappearing, causing a slow death of the profession. LLMs will probably never be good enough to do everything a developer needs to do: understand and improve or clean up requirements, consider future needs, test outside the box, evaluate performance and decide where and when to improve it…

Yeah it's a real catch-22. I think you're right though, LLM tech will keep up just enough to replace that lost workforce, till some day development as we know it will no longer exist, perhaps when the code becomes too abstract and complex for normal humans to ever hope to maintain without automation, similar to what happened to most assembly developers when the compiler came along.

Reply on news.ycombinator.com