And they will spend 12 hours trying to figure out which is the fake python library and the citation that the LLM has hallucinated from the real one. Vibe coding is just WYSIWYG on steroids in good and bad. WYSIWYG didn't go anywhere.
And they will spend 12 hours trying to figure out which is the fake python library and the citation that the LLM has hallucinated from the real one. Vibe coding is just WYSIWYG on steroids in good and bad. WYSIWYG didn't go anywhere.
Maybe you haven't used AI coding tools in a while, the latest ones can run build tools, write and run unit tests, run linters, and will try and fix any errors that may arise during those steps. Of course it's possible that a library may have been been hallucinated, but this will just trigger an error during the build job and the AI agent will go back and fix it. Same thing for failing unit tests.
Just last week I saw Copilot fixing a failing unit test, then running the test, then making some more changes and repeating the process until the test was running successfully. At some point during this process, it asked me if it could install a VS Code extension so that it could run the test by itself, I agreed then it went from there until the issue was resolved. This was with the bottom-tier free version of Copilot.
Of course there are limits to what AI tools can do, but they are evolving all the time and at some point in a not too distant future they will be good enough in most cases.
Regarding hallucinated citations, I imagine that the problem can be solved by allowing the LLM to access and verify citations, then the agent can fix its own hallucinations just like most coding agents already do.
Like MS Word?
These are pretty easy to solve problems tbh. LLM tools already exist that can work around “hallucinating libraries” effectively, not that this a real concern. It’s not magic, but these tired skeptic takes are just not based on reality.
It’s much more likely that LLMs will be used to supercharge visualizations with custom UIs and widgets, or in conjunction with things like MS excel for data analysis. Non-engineers won’t be vibe-coding a database anytime soon, but they could be making a PWA that marketing can use to add a filter on photos or help guide a biologist towards a python script to sort pictures of specimens based on an OpenCV model.
If you already know how to build software LLM are a godsend. I have actually had a quite a nice case recently when LLM invented some quite nice imaginary graphql mutators. I had enough experience in the field to not waste time debugging, a historian that hadn't shipped software before won't.
There were WYSIWYG before, before them was visual programming - we have tried to abstract that pesky complexity since forever. So far with no success. I don't see anything in LLM/Gen AI whatever that will change it. It will make good people more productive, it would make sloppy sloppier, it won't make people bad at solving problems good at it.
It's not that LLMs turn low skilled people into geniuses, it's that a large segment of even those with enough cognitive skills to work in software today will no longer have marketable skill levels. The experienced good ones will have some, but a lot won't.