I feel my main purpose isn't that I can technically do something my clients can't, but that I know how to interpret their needs and find the solutions that fit for them.
Now they just query some AI which spits out >some< solution and they won't know that they are wrong until it can't even be tracked back to their AI.
This will result in "but the computer said" excuses all over the place.
My example comes from maintenance in machine shops. My dad's siblings were almost all employed in machining in some fashion. One uncle was maintenance for a local large company. They decided they could move maintenance from in-house specialists and contract that work to a multi-state massive company; anyone can follow a maintenance manual right? All you need are the proper certifications and everything is cool, right? The cost was less, and the number of maintenance staff on site actually went up. Great for the owners.
But what they lost was consistency and understanding in the process that eventually led to closing the company. My uncle was there every day all year long for decades and could hear and feel in his feet when something was going wrong with a machine regardless of the maintenance schedule approved by the company that manufactured the machines. The new outside company didn't pay as well, and didn't have great benefits, which meant that they cycled through staff very regularly. No one was there consistently for more than a year or two. Even though they were technically following the manufacturers recommended maintenance schedule, machines went down. Machines then stayed down for days or weeks longer than they should've, because the expertise to repair either wasn't there, or the person leading the job would quit and they would have to cycle in a new maintenance person. They lost jobs, customers took work other places. 8 years later the company closed.
There are lessons in this kind of thing, but nobody learns them.
[0] https://books.worksinprogress.co/book/maintenance-of-everyth...
The other day I visited our old apartment and noticed the dishwasher was making loud noises. Worrying about whether it's about to fail and whether I should call in a technician, I just recorded a bit of the noise, screenshotted a spectrogram from an app, made some photos, and threw Claude at it. It proceeded to analyze the recording and match specific frequencies to components of the machine, and gave me an analysis indicating the machine is operating normally, and the noise is just the pump being louder than I'm used to due to machine being old and a bit worn out.
So we literally have that kind of capabilities in our pockets, and this will compensate for the problem of missing experience somewhat, even simply because unlike low-paid frequently-rotating staff, LLMs are not lazy and will do the detailed troubleshooting work even off a recording and "wtf?" as prompt, so if you can train your cheap, expendable staff to do just that, that's already supercharges them.
My point isn't that AI can substitute for expertise in general, not just yet, but that if you're competent enough to evaluate it, AI can absolutely do hours of work in minutes even for things involving troubleshooting on factory floor (or home appliances). And, that this is a lever where personnel on the floor may have sufficient expertise to evaluate diagnostic work, but not enough motivation or space to do the diagnostic work themselves.
Seems the AI analysis added no value, but took, up time...
People in other teams are slowly getting it. When I leave (or more likely "am let go"), the cold hard reality will become much more obvious during the next network problem ;)
I do think they are close enough to the stage where people will be replaced, even though it's not the right or correct thing to do, and that we need to move on from the "denial" phase quickly enough so our societies can adapt / regulate / legislate while not too late.
It kinda reminds me of the story of the Toyota process , while their car product lines were/are heavily automated the process still relies on the smalll teams on the line to spot/solve and iterarte over mistakes/inefficiences then add the refinements back onto the automated tasks.
Similarly althose stories of radilogists going jobless have not panned out , as interpreting slieds is a 'task' that a radiologist does.
Semantics , yeah but it does shape the way of thinkings regarding jobs/economy and not to mention the whole anthropomorphizing by the AI labs and boosters. In essence the wordings can/could introduce a healthy dose of reality/skepticism to the wildest predictions regarding job losses etc.
The kind of automation in software engineering already happened in finance yonks ago fellas - it’s called spreadsheet macros and presentation templates - they’ve been using boiler plate and filling gaps before you guys. Hello? Lol
Sometimes it’s in my favour. LLMs estimate very high prices and long timelines for software projects.
If it’s a legal / regulatory issue it’s usually wrong.
On a more serious note, the pre-LLM internet had very little useful detailed information regarding software estimation addressing effort for building something in specific domains.
Most writing on software estimation tried to tackle a general approach that could apply to all domains. So you had a "applicable to all, useful to none" situation going. And since crud apps were the most ubiquitous apps, software estimation techniques were perceived to give more accurate results with respect to those type of apps.
I'm sure internally big AI companies are accumulating vast amounts of estimation training data by documenting how long it takes their LLM to complete specific app building tasks. I don't think they would be in a rush to push that data into public models though as it would be like beating a dead horse that is the current human workforce anyway.
Without additional explanation this is indistinguishable from the role of a technical product manager.
Even in my personal life, while I've been addicted to using Claude and ChatGPT as a first step for research/projects/etc, I'm still not convinced they are actually saving me time -- just too many errors, too confidently stated, and I have to re-do all the research myself.
I wonder if the validation piece is easier with coding?
EDIT: To be clear, when I say I'm having trouble incorporating AI into my white-collar work, I'm talking about producing GOOD work. On a few occasions I have given in to the temptation of using Claude to produce work that looked good (superficially), without verifying everything.....it worked out, and I'm sure others are doing it. But that scares me, because it feels like a ticking timebomb.
Is it just faster? More persuasive? Better at coming up with novel arguments?
Speed is great from a client perspective, but not as important as accuracy on important stuff. I haven't seen LLMs produce arguments that are more persuasive than those crafted by good lawyers. Finding the "perfect case" in Westlaw is helpful, but rarely the deciding factor.
I could see asking an LLM to reproduce your work AFTER you've done a first pass, to see if you missed anything -- that seems like a real value add. But I truly can't understand how not using it would be "malpractice." In fact, I still think committing malpractice would be MUCH more likely with AI. (Now if by "malpractice", you meant pissing off clients with expensive bills, that's a different story...)
For example, theres a very popular coding skill set call Matt Pocock's Grill Me where it just asks you a relentless amount of questions about the feature you're trying to build. You may be able to tailor this somehow to act as the opposing counsel and have it poke holes in your argument. I suppose the closest analog would be a sparring partner.
Its helpful for me and my ADHD brain because I can go down all sorts of rabbit holes and it can easily keep track of all the ideas i have while grilling and it can send sub-agents out to investigate other claims to add more context to the grilling session.
For example, for your legal work, it can:
Develop a machine learning model (CRF, running on CPU) for clause extraction from scratch, Find training documents and training data and train the model for various provisions (you might help it here by pointing it to the directory with annotated documents),
Build an app where you upload your legal documents and it extracts the provisions you want from all documents.
Integrate semantic search and an AI agent into your app which lets it answer any question about your documents with exact evidence from the documents.
All of this from one well crafted prompt, a few skills and a workflow. It could probably complete this in one night, while you sleep.
Or you can point it to your saas of choice and tell it - 'create me a copy of this app'.
All you need to know is how to steer it through the planning/research phase.
So it's not that it's 'accurate' in legal terms, it's that it can build you any tool you want from scratch so that you can become more accurate.
Edit: add: I've found this to the degree that a relative who works at a top biglaw firm and has their internal AI (MS-based) and used to find ChatGPT and Claude actively time-wasteful, when I posed a question they'd spent a day+ fruitlessly researching (multiple areas of law and international), Claude came up with a very useful answer in ~10min — they were quite impressed.
The last year has made a huge difference
In my case, for some types of work - which sound similar to things you mention - I started slowing AI down intentionally. Software development, if it's something simple/common and thus "in-distribution", LLMs can zero-shot complex projects just fine. But in more R&D work, I found that errors crop up quickly and accumulate, and end result of "go and just make X" is usually bad.
Now, the problem I found with usual workflows for this kind of work is, agents race ahead and very quickly, I can't keep up anymore. So I reworked my workflow to make AI work in small steps optimized strictly for maximizing my ability to validate the outcome, and minimize time I spend doing it. So now, the thing (say diagnosing a performance regression of some complex codebase run on a cluster) may take me a whole day, where "unchained", agentic swarm would do it in 30-60 minutes - but, at the end of the day, my method is near 100% guaranteed to be correct, with every intermediate step vouched and reusable for future work, whereas "agentic swarm" approach, that hour of work, 90% of the times I'd just skim and throw away and reprompt, because I couldn't even begin to guess which part is correct, which isn't, so in the end, it was just a fast way to waste lots of tokens.
I see many people have complex systems involving skills, agent swarms with roles, agent factories, 20 termux panes and whatnot. Maybe that's the way, I don't know. Nobody knows. But it's not necessary to start getting value from LLMs. I therefore recommend my "lazy person's basic approach":
- Throw a task at LLM
- Check result, see what you like or not like about it
- If you just can't keep up, ask it to do smaller steps; if you don't understand, ask it to explain
- Just do that a lot, and don't be afraid to make both absurdly large and absurdly small asks
- You'll quickly get a feel for where AI can help you, and where it's not yet worth the time handholding it
I've used LLMs for some very small coding and math tasks, and I've found it much easier to work with on "formal" types of problems.
But for my consulting and legal work, which is "fuzzier," I struggle more with verification. I feel like I have to verify every single fact/premise in the LLM's output (this problem doesn't seem as big with coding?), as well as every single logical conclusion. Plus, even with explicit instructions I feel like the LLM is not great at anticipating what certain audiences will be interested in or find compelling.
Compared to the black-and-white nature of a computer program, all the components of a legal memorandum (premises, logical structure, conclusions, etc) seem so much fuzzier and require so much specific and accumulated context to apply reliably.
The output of agent swarms will of course grab attention, just like when you saw those same Microsoft Frontpage themes in the first few years it came out. But there's a deeper value in using the tool in harmony with your own experience and value additions.
I'd imagine just that could already be pretty useful for your line of work ? That does require to have a claude subscription and using Opus 5.5 for it to be worth it or cost efficient.
Then you can compound this: give it access to more of these docs, have it memorize, create skills on where to find what, then move on to more and more "vague" prompts and you'll notice it knows what to search for, where, and how.
Quickly enough you can ask it to generate an artifact explaining really complex stuff super succinctly and gain a ton of time. You can ask it for links to its sources in your docs for each point it makes in order to easily confirm it doesn't make anything up etc...
But to reach that point, you might need to go step by step so you know you can trust it because it didn't fail you on the smaller, easily verified steps you tried early on.
This is the level that many non-technical users are operating on.
There are only a few professions that are intrinsically precise with language.
No surprise we're seeing adoption in software programming and mathematics.
Since the vast number of potential users of LLMs are, as you say, poor practitioners of precise language, it is incumbent on LLM providers to continuously iterate on providing LLMs that work better with imprecise language.
Hence, the people who claim that precise language is some 10x - 100x productivty skill seem to ignore the cycle of big LLM providers who are optimizing the models for users who lack those. (One of the early optimizations was the chatbot asking follow-up questions to the user for example).
So I think the people who think that their precise language skills are going to allow them to avoid the jobocalypse are also in a separate bubble of theit own. Precise language skills are going to matter less and less as LLMs increase their hand-holding skills for less equipped users.
I have not come across a rigorously argued technical analysis that has claimed that using precise language can in any way eliminate the possibility of hallucinations or unaligned behavior. Short of providing a proof for the elimination of hallucinations and confabulations, merely reducing the behavior by a random percentage is not an acceptable status quo for completely removing the human.
It requires that LLMs are equally effective with imprecise language as precise language.
And while I'm sure the frontier labs are continually iterating on 'garbage in, something vaguely useful out', it's a high bar to get that output to be as useful as if it were more precisely prompted.
For regular work, since we use modern reasoning models, you might as well type your requests with the wrong keyboard layout selected.
It still decodes your intent and completes the requested task. You can work from your phone with voice dictation. It requires no precision at all.
So this, I think, is the crux of it.
When most people, at least in the world of SWE, think of what it means to be skilled at something, they think of knowledge and experience acquired over a much longer timeframe - months to years even. So from that pov, something you can get reasonably good at within just a few hours or days might not seem like much of a "skills moat" at all.
They are best placed to not fall for their own marketing claim that their tool can replace engineers.
1) They are knowingly lying about AI being at human replacement level rather than a useful developer tool
AND/OR
2) They think they are special, and that their software developers ("AI researchers") are doing harder stuff than those most other companies
Most people are wrapped in plastic instead of cotton.
There still seems to be a wide gulf in automation capabilities and things that require human dexterity, such as doing things for the infirm. Even just cooking and laundry seem quite a bit away. I guess the original tweet does mention this:
>Capability peaks in domains that are digital, verifiable and economically valuable.
Driving is the most impressive one so far, in my opinion.
They do rely on something that sounds like a COBOL mainframe from 40 years ago to me. They see people making the same basic clerical mistakes every year when towns and government organizations report their numbers. Like literally not being able to format numbers according to certain rules on the first try and such. Sending excel file in emails is the level of automation they rely on. They are all union jobs.
She was recently preparing for an obscure accounting certification. There's very little publicly available information to use and at most one textbook. She was asking her LLM (Gemini?) to formulate exam-level questions instead and explain the answers. She told me it worked spectacularly well.
But I assure you it's even "better" (not really but the task is so much easier) at making your spreadsheet look nice and compare actuals vs forecast for your budget than a complicated programming task. It will just one shot it, and it will be objectively correct, all in a very short time. But these non tech people don't know they can use an agent, they have asked one question in the chat interface with 0 context about what they're trying to do, and conclude they are not good enough.
I'm not saying everything is replaceable by AI - and by the way I don't want anything to be replaced by it, but I believe it will happen - just that some people don't realize yet how "trivial" their job is to modern models.
Just yesterday, my wife recounted to me a conversation with her friend, where the friend mentioned they felt lazy and had some piece of copywriting done by AI this time. To which my wife said, "TeMPOraL would've also had AI upload it to the website afterwards".
Yes, I would have, and I would've also had it install the tools needed to upload it too, because why not. The only difference between me and my wife's friend here is that I know I can ask AI that, and I know it will work.
The thing people are slow in realizing is that you don’t need to break the task down, and you don’t need to know how to do the task. You don’t even need to have the software installed on your computer to do the task, or even know that software to do the task exists - a sufficiently motivated LLM can pull down the needed open source tools and use them, or even build its own.
Two years ago, that was unthinkable. A year ago, it was a quick way to set a lot of tokens on fire. Today, well, I'm yet to find a thing Opus 5.5 doesn't do at least passable (and usually stellar) job on.
Tasks, oh yeah 100%.