but, are humans not also a magic black box? We don't know what's going on in other people's heads, and while you can communicate with a human and tell them to do something, they are prone to misunderstanding, not listening, or lying. (which is quite similar to how LLMs behave!)
> at the same time, i could totally see myself scanning through a README and going “yep, sounds like what i need” and making the same mistake (i need other people checking my work too).
yes, us humans have similar issues to the magic black box. i’m not arguing humans are perfect.
this is why we have human code review, tests, staging environments etc. in the release cycle. especially so in safety/security critical contexts. plus warnings from things like register articles/CVEs to keep track of.
like i said. don’t blindly trust the untrusted output (code) of these things — always verify it. like making sure your dependencies aren’t actually crypto miners. we should be doing that normally. but some people still seem to believe the hype about these “magic black box oracles”.
the whole “agentic”/mcp/vibe-coding pattern sounds completely fucking nightmare-ish to me as it reeks of “blindly trust everything LLM throws at you despite what we’ve learned in the last 20 years of software development”.
Vibe coding is all about deciding it doesn’t matter if the implementation is perfect. And that’s true for some things!
i was going to say, sure yeah i’m currently building a portfolio/personal website for myself in react/ts, purely for interview showing off etc. probably a good candidate for “vibe coding”, right? here’s the problem - which is explicitly discussed in the article - vibe coding this thing can bring in a bunch of horrible dependencies that do nefarious things.
so i’d be sitting in an interview showing off a few bits and pieces and suddenly their CPU usage spikes at 100% util over all cores because my vibe-coded personal site has a crypto miner package installed and i never noticed. maybe it does some data exfiltration as well just for shits and giggles. or maybe it does <insert some really dark thing here>.
“safety and security critical” applies in way more situations than people think it does within software engineering. so many mundane/boring/vibe-it-out-the-way things we do as software engineers have implicit security considerations to bear in mind (do i install package A or package B?). which is why i find the entire concept of “vibe-coding” to be nightmarish - it treats everything as a secondary consideration to convenience and laziness, including basic and boring security practices like “don’t just randomly install shit”.
I don't know about you, but for most people theory of mind develops around age 2...