Yoric

While I agree that handcrafting code has never been any kind of guarantee of secure code, I'd argue that it remains a condition: if all you have is a set of specs (or even worse, specs written by AI and not read carefully by anyone) and no mean to ensure that the implementation actually matches the spec, you essentially have no understanding of what the code is doing, which is security suicide.

Of course, AI reviewers and security tools will catch some security errors, but this should come as an addition to a proper security design, not instead of it.

I've started to use the term "downskilling" for what a number of companies are going through in the age of agents: layoffs of key employees in the name of increased efficiency, developers being incentivized to outsource their thinking when writing code, tests, specs, docs or reviews, agents increasingly replacing communication between team members...

Not GP, but please take a look at CloudFlare's data. As far as I understand, it was only measuring indexing bots, without attempting to look for anything political or shady.

I've been a moderator (on smaller platforms). Can confirm that it's standard.

> If AI found a way to exploit clinical trial design, it would be noticed and the errors corrected.

Do you think so? So far, AI is gaming lots of things, and I don't see much fixing in the processes.

Asking as a curious bystander: would that be sufficient?

I'm not familiar with ModernBert (my understanding stops around the original Bert), but it feels like this is asking it to do lots of heavy lifting. Can it do that much?

In fact, I'd argue that [vibe-]coding is the least interesting thing that LLMs can accelerate. I'm really frustrated because LLMs are insanely powerful, but also insanely misused.

> The more pertinent question in my mind is.. why does it still matter? Your customers don’t care about elegant code, they care about working products.

No, they don't care about nice code, but they do care about code that actually works. In my experience, there is a correlation between code that has been thought out correctly and code that hasn't, in terms of security, safety, compliance, data-safety, performance, etc. And so far, all LLM-produced code I've seen is severely lacking.

> That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.

So, I guess we agree on that :)

> It’s also just false. The good LLMs already produce output far better than the average programmer in most domains.

I hear this being repeated, but every time I've asked a LLM to come up with code without proper supervision, the result only superficially looked correct, but didn't stand up to scrutiny.

I wonder why our experiences differ.

Worth investigating, but I suspect that it's more complicated: as far as I understand, many billionaires are billionaires in valuation, but own relatively little currency.

I assume that GP speaks of reducing the salaries for C-suite to the levels they had before Reaganism? I personally suspect that this would be pretty good for companies.

Or perhaps they're thinking of nationalization, which leaves me a bit more skeptical.

> So I would say we are not normalizing failures (yet) but rather normalizing legacy.

Nice way of putting it.

Nah, jump straight to pi-calculus.

Unrealistic deadlines, productivity being measured in PR counts and tokens spent, and generally a 100% focus towards having a software factory produce all code.

In our latest evaluation, one of the criteria was whether you trust AI, with "trust" being defined as letting the AI write all the code, without reviewing it. And of course, not trusting AI meaning that you were behind the curve.

I'm confused. Aren't you actually describing racism?

Oh, if I had the ability to do these things, agents would be much nicer to use.

Sadly, at my current company, this doesn't seem to be acceptable practice.

> But you can still do that, much easier, with plain human words.

Not really? In my experience, to get anything precise done, you have to fight the LLM every step of the way. And then when you come back after a few days, you realize that it has overwritten the carefully crafted code or data structures.

> And who likes planning, architecting, directing a team, steering and giving advice, reviewing code, designing interfaces and APIs. Coding became more mentally exciting.

I love these parts. But in my experience, LLMs break much of that.

Planning/architecting? Great. So far, I haven't found any agent that actually follows the plans set out, though. They get something wrong, and it snowballs from here.

Directing a team, steering and giving advice? Absolutely. Some of my greatest achievements involve mentoring. But human teams learn from their mistakes, grow up and contribute insights. Agents don't.

Reviewing code? Well, maybe not so much fun, but it's usually a good way to understand what's going on, and to share experience. Except with agents, you need to spend most of your brainpower seeing through the misleading comments and documentation and choices and sycophancy, and the agent never learns from its mistakes.

Designing interfaces and APIs? Absolutely. Yet every time I look at code modified by my agent, I see that the contracts (internal or public) have been broken by the latest edits.

In my experience, LLMs can be very useful, for refactorings and as learning and review assistants, and sometimes as replacement for missing documentation. But codegen is the worst way to use them.

As far as I can tell from just looking at the documentation, yeah, it looks like a "better than nothing" and "definitely better than LLMs for a few tasks" tool, and quite possibly something that I'd build into a demo, but I'd be really careful deploying it in any kind of product where exactness matters.

It's funny, because in my experience, the process often looks like:

1. Human gives high-level design. 2. Agent generates wrong code with misleading comments. 3. In further iterations, agent get mislead by said code and comments, ends up generating insane workarounds.

It is.

Sadly, I know (major) companies that insist it's the process that needs to be solved, because it improves velocity (for some definition of velocity that involves dropping pretty much all quality gates).

Not only that, but even when you get the LLM to give you the right answer, if you ask it to turn that answer into actual code committed to your repo, it usually destroys something else in the process.

Yes, I've concluded a few years ago that VC money has been really bad for software quality.

I think that tinkerers is the wrong term.

I consider myself a professional. I lean towards understanding problems, making sure that they're fixed once and for all, and being able to foresee issues that the users will encounter and that product cannot see. And while I enjoy coding, code is secondary to all this. Yet I'm in Hell.

We're losing our ability (and time) to understand problems, we prefer churning out patches fast than actually fixing them, we delegate meaningful choices to an AI that has no ability to predict issues, we remove the opportunity to review and we end up leaving testing to the end user. As far as I can judge, we have stopped building products and are now shipping glorified demos.

Also, I can't speak for other products, but on the type of features I work on, AI is blatantly incompetent, but our C-suite just refuses to believe it, assuring us that it's just us having difficulties with the transition.

I don't know about other companies, but my company was to a large extent turned into a bodyshop after we decided to go all in into AI. We laid off / reorganized away the team, then amped up the deadlines and promises, leaving the handful of survivors to cover up for so many features and tickets that we have no time to actually work on quality.

And in our twice-yearly self-improvement reports, we have to write how we can use AI further. To reach grade 4/4, you need to "trust" the AI.

> Now even though Claude seems smarter than any person I interact with on a daily basis, it is all suspect.

That... is definitely not my experience.

I used Google when it appeared, and I'm pretty sure that I never heard these sentences uttered out. If anything, everybody was wowed by PageRank.

I seem to recall headlines about Russia infecting AIs with propaganda, one or two years ago.

I don't remember actually reading the article, so I have no idea how serious that was, but I'd be surprised if this wasn't being studied actively by all major countries already. And that may be one of the core reasons for the China vs US AI competition.

> Code review was difficult at the start due to the volume of code. One insight we gained midway through though is that the value humans bring to code review is judgment and business context. So we created a code-atlas skill that creates an artifact for PR reviewers. That artifact highlights the most important parts of the code.

Yeah, we did that, too.

But every time I end up, for some reason, digging up deep in the code, I realize that it's not nearly sufficient in our case.

Benchmarks look good.

Reviews? They were the first casualty.