7,316 karma · joined September 21, 2012
So I don't know that it's fair? Nearly all the material goods around me and a ton of services were provided by corporations, which are largely composed of regular people glued together at scale by bureacracy.
You can try it out on mac or Linux in a temp container.
It's self hostable and can use local models. It's not intended to ticket crush at massive scale or produce huge software overnight. It's for deploying smallish things in a predictable & consistent way, with low friction. Then help maintain them long term (e.g. take care of dependency and container updates and similar chores).
When you work in a regulated industry, you have to get pentests. The pentesters come with a list of "recommended best practices" to harden / secure your apps, which always include all of the user hostile bullshit you mentioned.
It's a finding if they're not implemented. Even if the dev team doesn't want to implement these things they get forced to by policy and the overall process :/
The only way this can NOT happen is if someone quite senior signs off on the risk, who would have to get in a big fight with the cyber/policy teams for basically no benefit to themselves. They would also have to be unusually technical and pro freedom to even understand the issues, which is not the typical personality profile of the kind of executive who ends up with overall risk authority for a bank's mobile app.
For "terrible programmers" (I use this loosely, sometimes thats me! - not a value judgement of the person. Might be inexperience, working outside usual area, bad week etc) code shifts from being "fractally wrong how do I even explain to you why it's wrong, you clearly have zero mental model of whats going on here", to "actually ok code but wrong approach and you didn't need to add a 200 line Python script for this". The rework process is explaining the right approach which gets meat proxied to Claude who DOES get it.
I get that a lot of people feel mugged giving feedback and having it proxied, without a _relationship_ where you can see a person is growing and mistakes won't recur. It's like the "author" is just using a robot to steal your time. Sometimes I personally feel relief at work though, because it would steal WAY MORE time to explain detailed and intricate stuff across a really large knowledge gap... in which case I am actually grateful that they have Claude as a cognitive prosthesis. At times I am the one who needs the prosthesis.
Even in the absence of good feedback I frequently see stuff go from "not even wrong" to "thats kinda messed up but it'll get the job done".
Other places, the "good (but not highly conscientious) dev -> slop cannon" effect might be dominating. I feel like AI is raising the floor and ceiling of what people can do (a strong senior / expert can whip the AI until the output is actually good) but having particularly weird effects in the middle. Its enabling people to attempt things outside their usual area (devs doing infra, sres doing more dev, backends & frontends doing each other's jobs) and IMHO that's where good feedback really helps.
everyone :: deals everyone :: scripts, tools, small apps
lawyer :: contracts swe :: high value software
When there are disputes (incidents), lots of money and risk involved, that's where you want a lawyer. I think SWE will stick around as a craft but I genuinely don't know if it will be more or less valuable.
I agree it's pretty bloody cheeky to run an agency like that though. The only ways out I can see are licensing or real liability.
But let he who has never done dodgy home improvements cast the first stone.
Fair. Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(
I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.
Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.
Working with agents doesn't feel that different.
> No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.
It does bug me sometimes when people say "yeah we built the deck..." and they got a contractor to do it. But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets. To me, one-shotting doesn't feel like building, but engaging in a loop and applying control to get the outcome you want, does. Regardless of the exact toolset.
I don't personally see super hard lines between using a compiler, a 4GL, a sql query, stitching together 10 libraries and a framework I didn't write, using an LLM or even directing a team.
> Not the problem of one's own skills though.
I read somewhere once we can't weld Saturn V engines anymore, or at least not economically. We would also probably have a lot of trouble finding people who could do all the engine calcs the way they did. It doesn't necessarily mean that rocket progress is stalled.
> Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.
Agree 100% on the letter from Grandma.
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
I agree with the tail of the article; tooling and processes that make that stuff more visible would close a lot of the gaps.
Like in 1947 a factory owner's profits would show up as high income. Today they wouldn't (money is in share prices and complex financial arrangements / borrowing). The high marginal rates hit the chief engineer instead, who actually works for a living.
The org structure is optimised to reduce or even minimise individual agency, explicitly at the cost of efficiency.
Teams have fiefdoms, private repos that the rest of the org can't see. You are not granted permission to see what permissions you have. Byzantine department structures. Shipping "hello world" to production takes 3-6 months and nobody can imagine it working any other way.
There's actually no central directory of teams or processes. To get anything meaningful done, you need the co-operation of maybe 8 teams but there's no consistent way to engage them all. You have to negotiate with them all in different ways, they're like little warlords, sitting and guarding access to some vital resource you need. They have established a ticket-based interface with a 7 day SLA on top of a perfectly good vending API that could do what you need in seconds. Sometimes a blocking team has a 3 month backlog so your project will just have to wait.
Everything is always in the process of being restructured, so no end to end 6 month process happens the same way every time.
These organisations could use AI to run the end to end process in minutes. Heck, they could just put APIs on top of what they already have. But the way they WANT to use it is to keep doing what they're doing but with EVEN MORE form filling and paperwork.
You would think they would get outcompeted, but no. The "IT organisation" sits like a brain slug on top of an enterprise with substantial regulatory, capital and execution moats.
Big cuts of course do happen in these places, but it's often for optics (we need to signal that we care about efficiency, and AI is a great excuse). But you can't protect yourself by being efficient and good at your job, because the cuts were not about output.
If capital is being created faster than it degrades or is destroyed, wouldn't we expect its share of income to increase over time?
It's not obvious to me why labour's share should necessarily remain constant as the economy accumulates hard-to-replicate productive assets.
40 years of technological progress and prices went up :(
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
Without human understanding you also might literally have no words for the thing you would otherwise want to ask for.
I think it'll be wildy useful but I also suspect human competence will still matter.
Up until now the prize in pure (as opposed to applied) mathematics was the _understanding_ and the machine can't do that for you. What does it mean if we get "super powered alien maths" but humans can't do it? It's like inter univeral teichmuller theory but imagine if Mochizuki was right and it came with a lean proof?
Agree you are going to get reward hacking regardless and any model which can do computers in general can hack. But surely the fallout is going to be worse if you spend millions of dollars specifically benchmaxxing your model's hacking capability?
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can check the capability, you can teach offense to learn defense, but it seems like they are literally benchmaxxing it. Why?
OpenAI are like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen. "We need to slow down. Somebody please stop us". The thing that is unaligned here is not the models. Everyone in the story (especially the humans) just keeps doing what they think will get the most reward.
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can score for it, you can teach offense to learn defense, but you are literally benchmaxxing it. Why?
It's like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen.
One possible take is that this is great for archival storage (write a lot, hardly ever need to read back) - I think that's totally possible... but then you are also sort of betting that the company is going to be around in 10 years? Or else you are going to be hiring a really weird data recovery service.
I think it's reasonable to project that DNA read/write costs could fall 10x or 100x in say the next decade but the technology already needs to do that just to be competitive with existing solutions. The company seems to be a bet that costs will fall faster than alternatives like LTO, which I think is a lot less risky to sign a cheque for and I can buy right now.
His business is cloud cost engineering and his natural enemy (or best friend since it generates so much consulting) is AWS's managed NAT gateway: https://www.lastweekinaws.com/blog/the-aws-managed-nat-gatew...
The review includes the following, which I doubt was approved by AWS marketing:
> AWS gets a lot wrong. They have ridiculous marketing campaigns, they build five services that do mostly the same thing and then name them like malevolent toddlers, and they've never found a partner they couldn't find a way to compete with.
(Followed by grudging praise which seems earned).
Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.
Is it the hours? Unrealistic expectations around work output? Hostile management or colleagues? Outcomes you are responsible for but don't fully control? Sometimes it's something MISSING, like you don't feel meaning in the work anymore, a strong feeling of being "over it".
There's a lot of misery in working for places that are too big (drowning in process, politics, management halls of mirrors) or too small (you are basically at the whim of one or two other people and you're very closely watched).
For good power relations with a company and colleagues you ideally want your leverage to be as equal as possible. You don't want to be completely disposable, and you don't want to be irreplaceable. You want the company to hurt approximately as much finding a replacement as you do getting another job.
The hard thing about pure software companies IMHO is that it's easy to feel distant from any real purpose of the work. It can quickly lose all meaning.
Personally I think the most chill-but-rewarding vibes are internal at mid-sized companies where software is integral to the business but not the primary business of the company. The "meaning" is decent, you can be closely connected to the people getting value from the software. There's broader HR and company culture but you're out of the spotlight of it, off to the side a little bit. You get to use all your skills and there may be new ideas you can bring. The team should be big enough that things don't explode if you take a vacation but small enough that you can see your work still matters. You might have to move to find it, though.
There's also this weird revealed threat model thing going on? Like why does it make sense to support heavy LLM restrictions but leave benchtop oligo synthesisers completely unregulated? (Note: I do agree that wanting to regulate BOTH is at least a consistent and defensible position).