It’s so odd to me how companies decided what LLMs are capable of without data backing it up. Were all the execs conned or something ?
It’s also worth mentioning that it still might be the right business strategy for some companies / industries. We are only 3 years into the revolution of AI for business processes and in previous revolutions there were riots, sabotage efforts, factories still being created in the style of the previous revolution, etc.
"everyone! ship ship ship! make production ready versions of what was triaged from the hackathon! nnnowwwwww"
"everyone! wow 80% correct, prompt engineer it to be stricter.... and with a bigger model! wow 98% correct! this whole division is made redundant!"
"everyone! its not 98% accurate and even if it was, thats a huge set of errors given our volume!"
"everyone! our AI bills have skyrocketed! they're charging us differently because we're an enterprise! kill the AI, kill the AI"
LLMs can write code. They're actually pretty good at it. So problem solved, right? Cost centre cost reduction. Bam!
In reality the more competent in the job were really good at understanding business problems and holding domain specific knowledge, working with the other people on the team to translate that into a problem a computer could solve, and with understanding and diagnosing what was happening in the broader system, not just in a "program."
Someone needs to write the prompts given to the LLMs and decide if what they came back with even makes any sense. Someone needs to respond to pages in the middle of the night. Someone needs to be able to look at the system and have a bigger picture understanding of how it fits with the business' needs, etc. etc. That's a software engineer.
I honestly think not enough in middle and upper management really understand what software development actually is.
If anything, I feel like AI has made domain expertise more important, not less, as the "confidently wrong" error case for agents has no one able to sanity check it. At least before AI a human would dip their toe in the water and usually realize that having no idea what they were doing, and not even being able to understand what the comments mean, was a sign that they need to go find someone more experienced to help.
Yeah, this is nuts because at every company I've worked at it's assumed that engineers are thinking about things like product market fit, how a feature would be sold/ the "value" of the feature itself, how we would support the feature (not just the code, but how support would manage it), etc.
I don't think people realize how much of a hand engineers have in these conversations because we don't champion that, but we think a lot about the product as a whole. Obviously we don't spend as much time thinking about how the product will be sold as a sales person will, but we absolutely think about it, in my experience.
We think a lot about the business, like a massive amount about the system as a whole across these organizational boundaries.
my company spends millions a year on tokens and when asked about ROI the CTO just says "LoC is up! LoC isn't a good measure of productivity but it's a measure, right? right?"
They delegate and hire subordinates to do a job. It is by design that the communication won't involve 100% of the work done
You hire people to do a job, not to be a remote controlled puppet
I guess its impossible for an executive to know ALL the details of the work they delegate, but I'd be willing to wager that executives who understand the details function better in the long run.
It certainly isn't tautological that executives be imbeciles about the businesses they run.
But that want is limited (needs to be), and also it depends on the IC to explain things
Thats partly why they get so far.