prerok

Could not agree more. I also taught for some time and generally had good results, almost everybody "got it".

There was one person, though, that I just could not get anywhere with. Even after several private lessons. Turned out that somehow she convinced herself that she will never get it and never be able to progress. Even if she did get it right one week, the next week was as if that never happened. I found no way across that barrier :(

Thank you for the explanation but I still don't quite get it. Is this code mounted to a separate VM where the agent is running? I mean, how does the sandboxing of agents really work?

The reason I am asking is because if it's not sandboxed on the OS level, then commands it runs may escape the harness sandboxing. Even more problematic can be a command added to some auto running script that will get executed at some point outside of the sandbox (when the developer is doing actions). So, reviewing everything before anything is executed seems like the only safe way to do it. What am I missing?

Hahaha, hilarious. I could also tell a story or two like that.

I have to say, though, I have no idea what the management is thinking when they hire such clueless PMs. Even worse, I have seen clueless product owners who had no idea about the domain we were in. I guess a recent example could be Ive designing the Luce.

Maybe I am just envious. Maybe I just wish I could BS my way through life like these characters do.

I've seen these suggestions but I am really curious about the set up because I just don't get it.

If you want to work on the code then you need to have access to the repositories, so you need the github token. Then, to test the app, you may need your own backend token. And VPN. Of course, only to DEV, of course all tokens encrypted. So, only DEV and your branch of the code is in danger. In my view, even that is pretty bad.

So, how does such a set up work?

Yeah, good luck with that. I've had too many libraries change underneath me by clearly violating the previous contract that I know it doesn't work.

Well, yes, but here the Pareto principle comes to mind, though.

Well, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn.

With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick.

What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowledge into words and therefore couldn't produce clear instructions for juniors to quickly ramp up. Now we are trying to do just that. I think we will find, just as we did in the past, that it's not possible. I do think a good harness improves results but LLMs will not be able to reach senior levels. Just my 2c.

Are we, though? No offence but if an external module starts calling an internal function they better prepare a PR that changes the internal (yes, I know, by convention) to a public one. This is a signal to the developers of that module that they have to maintain the behavior.

That PR might well be rejected. And you have to work with the module owners to get your case supported.

Anything else is not responsible and I would not call it "adult".

It is indeed a significant undertaking. But... it is doable. I've worked on a code base that converted several functionalities to golang. It did take a lot of effort and quite a lot of planning.

Indeed and using SIGKILL is really cruel. At least with SIGTERM the process can say its goodbyes. /j

No, then you go back to programming on the white board, just like in college. /j

Indeed, however I would also point out that senior engineers have already been expected to be good at reading code: they were expected to evaluate the code quality of other contributors, so they had to be able to do that.

In fact, from my personal experience, going from junior to mid to senior, that was the hardest thing. Reading the code and thinking if what they did was really correct and will not have additional undesired side-effects was hard to become efficient at (it didn't help that we were working in C back then).

So, really, I think that for juniors it's actually much harder because if they want to do due dilligence they have to do the same evaluation but without the years of experience working with that code base. I can understand, even if I don't like it, that they just submit the output of the LLM for the senior to review.

Exactly. To be fair, there would be people in both camps (all-in and no-go) and they would be the vocal ones.

Those are the minority. The majority will find the middle ground.

I've always felt we somewhat failed as engineers (I included, of course) when I was doing boilerplate by hand. We should have taken the time to get those autogenerated. But... it takes time and the generator is always more complex than the produced code, sometimes even by a few factors, so it also takes an expert to maintain.

As for tests, I've seen LLMs produce good ones as well as useless ones. I guess it's all about instructions... sorry, prompts... no, sorry, prompt engineering to get it right and done properly.

That said, I am also very concerned with brain rot. Engineers nowadays can commit code they don't understand without a blink of the eye. Slowly, the knowledge may get sparse if we are not careful about it.

Same for me. I have not yet successfully used an LLM to generate code to produce a feature, though to be fair, it might just be because I don't have the patience to go back and forth with it (will readily admit that I am using it wrong).

Also, while some code review comments are just plain wrong, LLMs did produce some damn good comments, on the same level as a different senior engineer might note had they taken the time to study the code carefully.

Even if the jury is still out, I would still say you are both right already.

The amount of slop produced even in company setting is staggering and I don't like it one bit that neither the submitter nor the reviewer of the PR paid due dilligence. And I am only complaining because it then becomes my problem. So, then I have to start nagging people to clean that up. I can say with 100% certainty that the problems I face now would not have happened without LLMs.

That said, used with care, with proper supervision, with dilligence to review what LLMs did, I still think they can be and are beneficial.

I think that we are just not used to getting results of questionable quality from the tools we use. So, I am hopeful that we will learn and it will improve with time but still find myself dreading the age of the vibe coder.

Ok, so an edit to the parent post cites my comment as an incorrect citation needed as if it's incorrect.

Another statement also claims that genocide was performed because it also includes religious groups.

None of that is wrong.

What is wrong is claiming that they actually performed genocide. They overtook, yes. Did not perform genocide. Even the cited article says, that "christian communities were dissatisfied with the change". If there was genocide, who was left to be dissatisfied?

Hah, that's not the norm? In my country it was. To be fair, the professors were required to give the students learning material in our native language and while some fields do contain other experts, the software field is different, so there was one book by that professor and that was it.

Most professors didn't mind how you got the material. But one of them... geez, every year he changed the content slightly and if you didn't have the latest one, he would write the test so that you would barely pass. The irony is that his lectures were really good and engaging but he really was a shitty person.

Citation needed. Since year 1300 or for 1300 years? The former is closer to the truth than the latter, AFAIK.

Don't forget the crusades.

Historians are nowadays equivocal in saying that the "dark ages" is really a misnomer. It's the middle ages and it was more marred (in Europe) by several powers warring with each other than by any religious "darkening".

It's a bit disingenius to equate spread of a religion to a disease.

Coops usually wouldn't be, but they almost all went under some time back where I live.

This. They will call up a high rank and tell them about it and then it would trickle down. It would not happen through regular support channels.

Why would it be a mistake? It's efficient for the target platform.

The same code can be compiled for different platforms, yes, but the assembly and machine code will vary significantly, so it could behave differently. Porting to a new platform was usually a very complex process, but the code produced was efficient. Nobody seems to care about this nowadays, though, it seems.

Somewhat true, but C is pretty close to translating directly to machine code, even if most compilers now do so many complex things the assembly can be pretty far off. My point being is that if you have a type int in your program, it's specifically tied to the byte size of an integer on the target platform. While it can be 8, 16, 32, 64 bits, it's defined based on what the target platform supports efficiently.

So, when you say, "it's purely the language", I have to disagree. The language means different things on different platforms but it's still defined exactly on the target platform. And it's efficient on that platform.

Nowadays, we prefer correct vs. efficient, which I do agree with, of course. But, I also understand why C is like it is. It is possible to claim it's a problem of the language but I would argue that it is not. C gives us barebones and working with it we have to know this. If that's not needed then sure, other languages will be easier to work with.

> crash in a C program that turned out to be due to the compiler removing a null check.

The what now? Though not lately, I did program in C for 15 years and never seen something like this. I did see some compiler bugs on obscure platforms (SINIX, IRIX, HPUX on Itanium64, etc.) with proprietary compilers, this kind of thing would make really get me shouting.

Were you able to determine why the compiler did this? Is it a bug in the compiler?

Yeah, it's a shame we never got something like boost for C. Every company I ever worked for had its own common C library solving these problems.

His point of origin is immaterial to my point. I was not commenting specifically for Mr. Karpathy. My point was generic.

I would not say that this is because of this point. It's because investors in Europe are more conservative. Employees are as well, it's true, so it's strange to have someone out of the norm. It's not a red flag per se, but it's a thing to be evaluated.