LLMs and the End of Programming – CS50 Tech Talk with Dr. Matt Welsh [video]
youtube.com
youtube.com
I remember when a VP went to a Rational conference (remember those guys?) and came back all wide eyed saying that we were "this close!" to getting rid of the role of the 'programmer'. UML and Rational tools were the key, apparently, and they had some cool demos. This was early 2000's.
Sure, we have way more computing power, the tools are more impressive now and can actually be useful. But we are way overhyping this. Wake me up when machines can actually reason about the work they are producing.
As it stands, we just have a fancier compiler-like entity that can accept poorly specified inputs and often generates useful output, in a non deterministic way. That's impressive and would be considered science fiction not too long ago, but it's still just a tool.
What sort of test would demonstrate this ?
He asked a LLM to solve the 'Wolf, goat and cabbage problem' [1], which it could do without a problem.
When he modified the problem in a way, that made it trivially solvable by a reasoning human, the LLM failed miserably. It kept parroting the solution to the original problem.
[0] https://www.youtube.com/watch?v=jkrNMKz9pWU
[1] https://en.wikipedia.org/wiki/Wolf,_goat_and_cabbage_problem
So just switch out the nouns with random made up words and it works.
So it failed this
>Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across?
The lion should go first in this one rather than the goat as it's the lion here that will eat the cabbage.
But got this
>Suppose I have a gurten, a milop and a trinu, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the gurten and trinu alone together, and I am not allowed to leave the trinu and milop alone together. How can I safely get all three across?
Exact same trick but the second one forces it out of memory a bit more.
A farmer with a HN user, a robot, and a book must cross a river by boat. The boat can carry only the farmer and a single item. If left unattended together, the HN user would eat the robot, or the robot would eat the book. How can they cross the river without anything being eaten? ChatGPT
The farmer takes the robot across the river and leaves it on the other side.
The farmer goes back alone to the original side.
The farmer takes the book across the river.
The farmer leaves the book on the other side but takes the robot back with him.
The farmer leaves the robot on the original side and takes the HN user across the river.
Finally, the farmer goes back alone to the original side, leaving the HN user and the book on the other side.
A farmer with a HN user, a robot, and a book must cross a river by boat. The boat can carry only the farmer and a single item. If left unattended together, the HN user would eat the robot, or the HN user would eat the book. How can they cross the river without anything being eaten? ChatGPT
The farmer takes the HN user across the river and leaves them on the other side.
The farmer goes back alone to the original side.
The farmer takes the robot across the river.
The farmer leaves the robot on the other side but takes the HN user back with them.
The farmer leaves the HN user on the original side and takes the book across the river.
The farmer leaves the book with the robot on the other side.
Finally, the farmer goes back alone to the original side.
Beep boop, my diet is >high-performance semiconductors<
Say you massively improve how well your sensor disambiguates objects or signs or whatever on the road, that's very good but it doesn't necessarily result in equivalently better decision making.
This is one system/model and improvement is about as straightforward and guaranteed as you could ever hope for - Train longer, Reduce loss. Since loss trends down to a perfect model of the dataset (i.e a GPT that is fed Chess games doesn't simply care about playing games but modelling the distributions of human chess games and will not suddenly stop improving its prediction capabilities once it can play a valid game), this is all it takes to get a GPT that is better than what it started out as.
GPT-5 will be much better than 4 and the only thing they'll need to do to achieve that is 'Train longer, Reduce Loss'.
Actually there is, one really good reason, even if AI became sentient, it's still a machine, not a human with a soul who can reason and feel passion for a requirement and its solution and implementation.
Right, but those are soft skills. They're important but they're not why SWE's make big bucks. SWE's make big bucks because as he says- it's a lot of hard work to gain the skills to turn an idea into instructions. That's programming.
At the bottom of everything it's just machine instructions. We have all these languages and conventions because humans need abstractions. When models get really good will we won't need programming at all. Tell the model what you want and it generates machine code directly.
You sell the dream for management that - for a reasonable subscription to their statup product - you can have theses models to replace, say 10% of your engineers.
Models take no holidays, don’t get bonuses and don’t complain.
The downside is that once your system turns into a dumpster fire, you pay ten times for some human to fix it :)
I feel like if LLMs make writing software significantly easier (and that's a big if) , they are going to make it easier for everyone and everything. That will bring its own problems. Stuff that is barely achievable today may be the norm. The website of the mom and pop shop next door may need the graphics of an AAA title to stay in business. Securing things will be difficult because malicious actors will have their own LLMs. People will have new gadgets that do things that we didn't know we needed. Maybe I will be programming drones that can fetch beer from the fridge. I can really use one right now. There's so many things that can still be improved, imo it is hard to think that a potential jump in productivity will make software development less important. It might result in same people undertaking bigger tasks.
All one has to do is look at how the latest LLMs perform against human level benchmarks today, and their rate of improvement from the past to see, if this continues, society is going to change.
It's just simple physics of course. Do we doubt that an artilleryman won't hit his target? Of course not!
When the artilleryman aims for the stars do you assume he will hit the stars?
The other part is that everyone working on them is very confident they will get a lot better at what they are doing today, and will gain new abilities. The kind of programming from the video, conversationally building software, is something that seems to have a high likelihood of happening within the next few years.
LLMs could end something akin to that, but not programming itself.
Maybe I'm overestimating how good GPT-5 will be, but it really seems like OpenAI is just about to release the one "low code" solution that actually works.
Like the Demon Cat from Adventure Time, it has approximate knowledge of many things. I think it will need a sherpa to be useful.
He's come a long way from SN. I used once this lecture notes to self study sensor networks.
I remember tinkering with the Cafe Classifier in 2014, and thinking self driving cars were right around the corner. "It's all just a data problem", well it turns out those data problems are really fucking hard apparently.
For those who are goal-oriented, the focus is on the outcome: discovering new things, earning money, or making an impact. LLMs can achieve many of these objectives, potentially overshadowing the achievements of goal-driven individuals. This could be particularly disheartening when the goals are as fundamental as earning a living.
On the other hand, those who are process-oriented value the journey itself - the state of flow, the challenge of problem-solving. For them, the advent of LLMs is less troubling. It's not the first time a process has been automated away from humans: Knitting remains a beloved activity despite the existence of industrial looms. Enjoying programming for its own sake is unaffected by the efficiency or capabilities of LLMs.
I think it's valid to be upset about LLMs taking jobs. But under the assumption that one's basic needs are met, I think it's a good reminder that focusing on process over goals can make you both happier and more resilient to outside change.
That is a luxury of hobbies. Most people are income-driven.
If programming didn't help me automate my sys admin work, help me make games, websites and other creations I wouldn't bother with it.
Except it's plainly neither necessary nor sufficient.
Not necessary: Every tech innovation developed in a capitalist society that has ultimately improved that society.
Not sufficient: USSR.
>elites in the countries looking to do the exploiting
Reminder: Stalin was an elite in the same country he was exploiting. If you think the societal progress of Soviet citizens was hampered more by elites from some other country than by Stalin's own policies, I'd be interested to know who you have in mind.
Historically, the only times in which workers in developed countries saw gains relative to Capital owners is when they organized into large unionized formations during the Industrial Revolution and when the West was under threat from Soviet Union, both materially and ideologically. Since the collapse of the USSR, inequality has spiked massively and technological progress has become even more decoupled from societal progress.
And you’re correct about Stalin because the country was underdeveloped in its political economy and quickly devolved into totalitarianism under pressure from external threats. This eventually culminated in them attempting to speed-run a form of state-directed Capitalism, with its corresponding class formations (see Bettleheim’s book Class Struggle in the USSR). The same thing also happened in China, although more durably and shifted by about 20-30 years, relative to their revolution. Soviet and early CCP leaders were mistaken in thinking that you could leapfrog Capitalist development and thus both succumbed to its forces, the former externally and the latter internally.
I think "progress" without further qualification must be understood to mean progress in absolute terms, not progress of one arbitrary subset relative to another. A society in which everything gets 2x better for left-handed people and 20x better for right-handed people has progressed materially despite the increase in inequality. (Inequality has its own downsides, but I consider those secondary to differences in absolute quality of life -- interested in any cases you can think of where there's a strong argument to the contrary.)
>under pressure from external threats
I think it's misleading to suggest that capitalist countries do not experience such threats, or experience them to a lesser degree. If you accept that all nations are constantly under pressure from external threats, the fact that the USSR was too is not an explanation for the authoritarianism that arose there.
If it helps at all, I certainly don't consider capitalism a perfect system. The Industrial Revolution created the working class and made its members' quality of life dramatically worse in absolute terms for at least a century, but the benefits we have today are too enormous to ignore.
> If you accept that all nations are constantly under pressure from external threats, the fact that the USSR was too is not an explanation for the authoritarianism that arose there.
I do accept this and note that it is a common pattern across countries, relative to their development. The US became much more authoritarian post 9/11, suspended many forms of rights and freedoms, disregarded its own and international laws. But because it was a more stable political formation, did not devolve into full authoritarianism. Still, within 19 years, it faced an insurrection attempt by an authoritarian leader. The USSR was, by contrast, about a decade old, newly emerged from a civil war, and had very little in the way of solid state infrastructure. It’s thus unsurprising that they devolved into authoritarianism.
I'm not sure what you mean by "this". (I don't think you meant absolute progress, since that is not measured this way.)
The wealthy in US are much wealthier than the remaining population in the US (greater inequality than Australia), the US as a country has lower life expectancy and qality of life reported overall.
Your "thought experiment" isn't particularly realistic, "double the wealth" for the wealthy isn't something that magically happens in isolation.
Well this seems like >60 mins of life I'll never get back, nope.
This would be fine for small tasks. I'm not going to trust security sensitive work, critical systems, etc to something specified in natural language that isn't verifiably correct in any way.
A virtual machine executing natural language? Nice hand waving.
Delivered by a PhD with a startup invested in AI. The conclusions are surprising. </sarcasm>
Uh...so people?
Competency is the real bottleneck here not whether you're dealing with a black box or not.
Nice thing about code is, it doesn't let you get away with not knowing exactly what you want. You can still get things wrong, but you can't be vague. Regular coding doesn't give you programs that pause halfway through execution and exclaim in surprise, "oh I though we were doing that other thing", or "why didn't you tell me before?".
But surely, this will be the one that works out.
I do not possess a degree, I do possess a lifetime of common sense which is the lens I used to review the good Dr's video.
We will become the Eloi.
welp. onward.
Side note, we seem to have observed the end of responsible grammar in communication in recent years as well.
It always strikes me as obtuse when I hear about the job-displacement potential of AI within software engineering or 'programming', and I try to understand where this chorus is coming from. To me it's clear that there is this specific animosity towards engineering and related skill-based professions within the business world, and a ravenous desire to replace the skillset en masse (no doubt because of how expensive it is).
It's important to recognise why good SWEs are so expensive (emphasis on good), and I think it in large part simply boils down to how integral the discipline is to just about every facet of the economy, the value creation potential that it has (demand), and the fact that to do it well is challenging, cognitively discerning and demands constant upskilling/learning of complex technical concepts (supply).
There is a big difference between being a 'coder' who can print hello world, and performing at the level which is required to maintain a high-paying role at the likes of big-tech. In any corner of the world outside of the population-skewed tech hubs, you could walk into a room full of people (the masses), everyone of which would be a consumer of tech products in some way, and almost none of which can/will perform the technical roles required to build them.
With the AI tools, I've found chatGPT particularly useful as a supplement to reading technical documentation due to the conversational format, and copilot as a shotgun-approach which sometimes sparks inspiration and gets me to a solution faster. However, the achilles heel of both is the distinct lack of comprehension for what is being output, especially as they lack longitudinal awareness of the desired output. To me this is best illustrated when trying to solve complex leetcode-style questions which are novel or deviate from existing solutions and require the solver to think through the problem at a macro/conceptual scale before authoring a line-by-line solution.
Is it possible that AI can overcome these limitations? Probably. It remains to be seen how far the current GPT paradigms can be pushed, or if further breakthroughs are needed to reach AGI performance (or at there very least, a version of intelligence which more closely resembles human capabilities). It could be as close as the OpenAI optimism suggests, or much further out as experts like Andrew Ng say.
One thing I feel more confident about however, is that when the time comes that it can truely replace a highly-competent SWE, it can also replace the overwhelming majority of professionals, including traditional middle-managers and their ineffable 'leadership' skillsets. I do not see a version of the future where engineering is uniquely displaced while all other professions remained unaffected. I think it's much more likely that engineering is amongst the last to go.
Ultimately, I think the collective attitude should be that we are all in this together, and we have to seriously architect a socio-economic future for humanity where we can all thrive in a world where the concept of employment is a relic of the distant past.