It’s not just code. My wife is a physician and I got her to do a few medical prompts with ChatGPT. The output looked correct the me, and if I read it somewhere I would completely have accepted it. But she could point out numerous severe flaws.
It’s not just code. My wife is a physician and I got her to do a few medical prompts with ChatGPT. The output looked correct the me, and if I read it somewhere I would completely have accepted it. But she could point out numerous severe flaws.
What the new generation of models is showing us, and what we've been gradually seeing over decades of AI research, is that there is a lot more tedium and a lot less creativity in our work than we thought. Look at how much this repo got right, even if there are bugs and nonsensical sections (I didn't see any upon quick skim, but I've also never implemented my own language before). It would take so much less time to debug this program than it would to type all that code from scratch. AI consistently turns out to be a human effort accelerator, and not a human effort replacer.
Maybe in the future it will be good enough to replace humans, but even what is now tried-and-tested technology like machine translation, spam classification, and object detection in images still requires a human in the loop to double-check the results. I'm pessimistic about the AI takeover, but I'm becoming very optimistic about AI precipitating a small industrial revolution. My only concern is about who gets to reap the benefits, but that debate predates the first industrial revolution.
Digital software is merely entering a realm of algorithmic (d)efficiency at least as old as biology, morphogenetic software: so long, be gone abstract truth table resilience unable to detect a shirt without stripes [4], welcome gradient exploration and error minimization able to give the synthetic mind, which, similarly to the carbon-based mind, will make ridiculous errors, just look at a child failing to walk [5].
[1] Ames Window https://www.youtube.com/watch?v=0KrpZMNEDOY
[2] https://www.researchgate.net/publication/259354882_The_Mulle...
[3] https://en.wikipedia.org/wiki/M%C3%BCller-Lyer_illusion
[4] https://github.com/elsamuko/Shirt-without-Stripes
[5] https://media.tenor.com/uB5ijGdseFwAAAAC/stumble-haha.gif
For knowledge derived responses, the questions require (a) expertise in formulation, (b) expertise in interpretation.
Also turn down the temperature, while keeping repetition minimal and token reuse low, to explore closer to its latent correlations.
I think we might need to start designing PLs with AI generation and processing specifically in mind. Explicit contracts everywhere might be a good starting point.
It left off some necessary vital signs from the scenario, then in the answer concluded that the patient had severe pneumonia and should be treated with a specific antibiotic.
Based on the prompt, the pneumonia probably wasn’t severe, antibiotics weren’t warranted, and even if they were, the antibiotic specified was completely wrong.
It looked fine to me—completely plausible. It took an expert to spot the flaws.
She also asked it to explain the work up for a febrile neonate. The answer again sounded plausible to me. But she said it sounded like a clueless med student trying to BS their way through an answer.