Comment by mschuster91 | Hacker News Reader

Comment by mschuster91 | Hacker News Reader

Parent Full thread

> AI now leads us into some new movement, the worst version of which a few model vendors protected by regulations absorb almost all technical accomplishment and lease it back out to everybody else, and the best version of which democratizes technical construction so widely that individuals can naively spin up effective solutions for the most idiosyncratic needs.

Yeah and the issues from these vibecoded one-off programs will be felt for generations.

I got out of IT partially because I could see the writing on the wall. The "load bearing Excel sheet" is one thing, manageable if you could catch the person who wrote it in time before they depart or die and know your way around Excel - there's only so many ways to achieve a desired goal (unless the author used VBA, at which point all bets are off, and I member encountering a spreadsheet that macro-called in a DLL shipped together with the sheet).

But vibe coded AI bullshit? That is supposed to be hosted by someone and kept secure (i.e. isn't hackable in a few seconds by anyone who manages to compromise some dingus's laptop? Oh hell no. I'm outta there, I'm too old for this crap.

>and I member encountering a spreadsheet that macro-called in a DLL shipped together with the sheet

That sounds absolutely horrifying. I never encountered one of those before thankfully.

Oh it gets better (not the OP but I think the following is even worse).

That DLL hooks into SCADA and sometimes sends updates to a PLC controlling safety critical equipment. When does it send updates versus just query the controller? No one knows. The guy who wrote it died in an (unrelated) industrial accident involving a runaway forklift after leaving the company. Even so much as debugging it would require shutting down the entire factory and replacing all the crucibles afterwards ($$$).

> When does it send updates versus just query the controller? No one knows.

Ask an LLM to reverse engineer it and explain it.

They said this:

> Even so much as debugging it would require shutting down the entire factory and replacing all the crucibles afterwards ($$$).

I'm guessing the fundamental reason debugging would require that isn't something an AI could get around. My guess is the LLM would immediately disregard whatever instructions you gave it not to cause having to replace all the crucibles, then say "Oops! You're right." when you point out that you asked it not to do that.

Hmmm, but shouldn't there be some way to just make a copy of the file, put in a container and tell the AI, scan the environment, do nothing except watch what the excel sheet does and record it, then determine how to create an identical sandbox containing the same outputs and feedback. Then see if the sheet works in that sandbox. If it does, then debug it until you get identical behavior out of a new program. Then have a week/month of dry runs just copying the inputs from the real world to it and capturing its output to compare to the dll. Do that for long enough and you should have a drag and drop solution right?

I'm not saying it's easy, or worth anyone's time, but it should be possible right?

> Hmmm, but shouldn't there be some way to just make a copy of the file, put in a container and tell the AI, scan the environment, do nothing except watch what the excel sheet does and record it, then determine how to create an identical sandbox containing the same outputs and feedback.

AI will attempt to evade your sandbox.

The only thing you can do is ask the AI to do a decompile and have it summarize, or better, do the summary yourself.

He used the wrong word, he means a mock or testing harness or a digital twin of the factory.

Thankfully we are finally after all these years getting regulation on that sort of stuff. It seems EU was 20 something years late... And no I really think that regulation in case like this is really reasonable idea.

That's even worse than the time I was on a conference call with the COO of a Fortune 500 retailer who referred to "the Excel spreadsheet we use to keep track of the stores we own".

The subject matter was building an auditing system to comply with the requirements of an IRS consent decree as part of a settlement for shoddy accounting practices which may or may not have included tracking billions of dollars of assets with Excel.

what would be the proper way to keep track of assets? I'm not financially literate at all, just curious

sadly this is coming to your linux-distro in no time! Far too many single-dev core building block which are now adding features at breakneck speed conversing with AI only.

> But vibe coded AI bullshit? That is supposed to be hosted by someone and kept secure

It's also astonishingly stupid. I don't want a one-off program for everything. I want a well-designed and battle-tested program that I can extend with AI, and that has a ton of people's AIs directed at it, constantly finding bugs, posting them on the bugtracker, and writing and debating pull requests.

I can't comprehend people who think that everything should be built over and over again, and that that will be safe, practical or cheap. It's like they're telling me that I should design a new language for each conversation that I have. Why would I?

The worst part is how far the capabilities of AI are from that dream anyway. They can't even write readable, intelligible code. They're being celebrated for being able to notice buffer overrun errors in code that people would also notice if anybody had ever really looked at some particular span of code in the last 10 years. It's finding errors in move fast and break things code and code that has been maintained by a weirdo professor in his spare time for free and half the economy somehow relies on. That's not magic, its a sign of how sloppy things got.

Because even with AI I don’t have infinite capacity to make every piece of software well designed and I have other deadlines.

>They can't even write readable, intelligible code.

Skill issue.

Because IT/programmers/tech folks in general have abdicated their responsibilities for too long, imo. The industry collectively started huffing their own farts and forgot they operate in a support role for most organizations.

The state of IT has been stagnant at best for the past 20-30 odd years. The promises of the 90's or earlier that we were going to have average office workers being able to automate their workflows never was realized.

Today with AI I'm watching random sales people automate a large portion of the bullshit part of their jobs IT and technology systems foisted upon them for no real reason. Those CRM systems that force you to click 28 buttons per lead you enter into the system, with all sorts of amazing UIx like having to restart from scratch if you miss a field, get logged out during data entry, etc. This is the standard state of most software people interact with on a daily basis.

If current IT didn't absolutely suck for most people, and hadn't become a means to it's own ends I think you'd see a lot less pent up demand for these vibe coded one-off applications that solve a specific problem for a single user.

It's been interesting to watch. Normies can finally get stuff done without the IT and developer gatekeeping and get on with doing the meaningful bits of their jobs. For better or worse. It's certainly going to blow up in quite a few cases, but I think the cat is out of the bag if progress in tooling - not even models - continues at a rapid pace.

This sounds exactly like the promises of putting a PC on everyone's desktop rather than a terminal connected to a locked-down mainframe, which in the end led to the lockdown migrating to PCs and the few employees who had any interest in automation forced to use tools like Excel because it was at least an improvement over VBScript and batch files and IT wouldn't let them install legit development tools unless their job description required them.

What are the odds that the average corporate IT department is going to allow the use of anything other than locked-down corporate AI with strict token quotas and stricter acceptable use policies?

> The industry collectively started huffing their own farts and forgot they operate in a support role for most organizations.

And most organizations still treat IT as a "support role" which means "cost center", only changing attitude when they inevitably get hacked, and a few years down the line, when enough of the managers who were there during the Great Hack are gone, IT gets the cuts again.

> Today with AI I'm watching random sales people automate a large portion of the bullshit part of their jobs IT and technology systems foisted upon them for no real reason.

Sadly that doesn't help at all. Random people automating heaps of crap in order to do their actual work, just makes the crap easier to manage for them. It doesn't remove it.

I'm reminded of health workers or teachers complaining that half their time is wasted on various types of record-keeping, because employer wants to record, check & micromanage everything. Similar with tax systems becoming ever more complex. Those are just a few of many examples.

AI assisting with managing that, just puts some $profitable$ control knobs in the hands of a very few providers. While at the same time enabling the mountain of crap to grow even further.

A real fix would be to use AI to get a better understanding of the existing heap of crap, and use that understanding to cut large chunks out & re-organise the remainder into a simpler stack.

Doesn't seem to me that that's being done. So far AI is mostly used to ADD to the crap-heap, faster & easier.

> A real fix would be to use AI to get a better understanding of the existing heap of crap, and use that understanding to cut large chunks out & re-organise the remainder into a simpler stack.

The thing is, that's been known for a long time, at the latest since the release of Bullshit Jobs [1].

Unfortunately, too many too rich interests are tied up in the status quo.

[1] https://en.wikipedia.org/wiki/Bullshit_Jobs

First, you do realize that when in some your-employer's-boring-CRM-system a certain field cannot be edited until a specific 5 conditions is met and that conditions allows an object to transition from "new" to "processing" status before field become available etc. then there was a reason for this.

Before Claude these people used to automate their processes in Excel. Then new people would come along and say, "None of this makes any sense to me so I'll just build my own even better automation in Excel". And again, and again. "Random sales people" won't ask Claude:

- Find out who was the PO in 2019, analyze why it was decided to introduce an additional status in FSTP-14475 instead of (...) What happens if the driver currently outside mobile coverage area and the task processing will return a 202 instead of a 200 (not described in the documentation; refer to FSTP-14385 for additional information), but the status will be updated before the deferred change returns (...)

Instead, they'll do something more like:

- I need to be able to quickly change this and that right here, get it done, don't make any mistakes.

Would be good if Claude could recognize an XY problem and respond "Understanding what you don't want is even more important than understanding what you do want" or "If you try to cut corners here again instead of following the established process - thereby making things more difficult for the people in the warehouse - I will inform your manager" instead of just charging ahead "you're absolutely right, see the goal, ignore the obstacles".

Second, it is easier to fix a flawed but already functional system than build a perfect one from scratch - without running into a problems, contradictions, previously unknown nuances during implementation. Because these contradictions, problems, and oddities over the last 10 years naturally gave you the requirements and specifications that no one of the sales-people and decision-making-people were able to articulate in a coherent way. And yes, now, having these specifications, you can rewrite it in Rust and do things better.

> They can't even write readable, intelligible code.

That really is no longer an up to date view of LLM's. Frontier models can and do write readable, intelligible code.

They can also write shit code.

Kind of like human devs.

> Frontier models can and do write readable, intelligible code

The problem is that even frontier models still are prone to regurgitation. They are not that capable of organization and relevance, which is part of why so much LLM output is insanely grating to read.

READMEs in particular are the worst. LLMs tend to regurgitate stuff like "what is located where" that is of zero relevance to the reader, especially if it's yet another bog standard Java, Symfony or React app.

Reply on news.ycombinator.com