Comment by myself248 | Hacker News Reader

Comment by myself248 | Hacker News Reader

Parent Full thread

It's appalling that computing in general, and unix in particular, seems to have this habit of intermingling payload and overhead.

It's like in-band signalling in the telephone network, where if you whistled the right tones into your call, you could affect the way the network processed said call. Except Ma Bell responded to that system being exploited by designing a comprehensive overhaul of the way signalling was handled, and spent a squadzillion dollars upgrading millions of tons of switching equipment to categorically exclude that entire class of attack from ever being possible.

Software, on the other hand, would need to replace no equipment whatsoever. Existing processors are perfectly capable of running code that handles the length of a string separately from its contents. There are existing languages that do this, they're just.... not used. String escapes and buffer overflows exist, going on decades now, due to nothing more than laziness, inertia, and negligence.

> It's like in-band signalling in the telephone network...

The major LLM providers claim that they are very committed to security and that their products are very dangerous and capable of great harm.

Given that their tools do what Ma Bell's systems did in the mid-1900s, [0] -which the entire software world relearned was a terrible idea by the early 1990s- they are definitely

1) Lying about the extent of their commitment to security

2) Lying about the extent of the harm that their tools are capable of

3) Both

My money is on #3.

Because of the fact that -in the absence of unambiguous laws that require meaningfully-severe punishment- even the most fucknasty and amateur hour security failures nearly always have little to no impact on the company that causes them, the major LLM providers have absolutely done the smart thing by providing commercial tools that have remote code execution vulns that would automatically give them a Critical CVSS score.

To put it another way: "the market" has no idea how to evaluate computer security claims. Because of this, every dollar you spend on proactively fixing security problems is nearly always a dollar wasted... it's better to wait until someone important gets Big Mad before spending the money. Does this make the world worse? Absolutely! Does this make companies selling software and software services much more money? Definitely!

[0] If the major LLM providers did separate unsanitized data from program instructions and ensure that the two are never mixed, things like [1] would not be possible.

[1] <https://www.schneier.com/blog/archives/2026/08/prompt-inject...>

And now the "AI" industry is doubling down on it - with hatbot prompt injection.

Reply on news.ycombinator.com