I’m not sure why the fact they formerly had FAANG jobs means they’re unworthy of sympathy for being laid off or why it means that they must be financially secure for life. We can sympathize with them and with other people that are even worse off, it’s not a zero sum game.
I'm 45 and ~21 years into my post grad-school career. I'm finally accepting that I will probably _always_ feel like i'm failing and doing things badly and that's OK. The evidence suggests that I have in fact been doing at least a good enough job in my employers eyes to sustain a career and progress. When I look back on this evidence I don't however look at it as a list of successes, I still mostly just see the things that could have been better. But by accepting that this is just how work feels I am relieved of the stress and anxiety that feeling has caused me for two decades.
It’s less profound sounding if instead of talking about “representations of the world” you talk about compressed representations of data or decision boundaries in a classifier over data.
I don’t think the good programmer bad programmer framing is helpful. I think AI fits the working style of some developers better than others. Where it fits I’ve seen some developers become so productive that they suck all the air out of the room and drown everyone else in review - not clear if this is good or bad. Where it doesn’t fit I’ve seen developers shut down because they find the workflow so depressing and their former productivity craters. Similarly I’ve seen perfectly good developers turn into slop generation machines and go largely unchecked because everyone else gets alarm fatigue from reviewing their PRs.
The first time I heard this recording I basically got stuck in a loop of listening to it on repeat for a couple of weeks. I just couldn’t comprehend that this was improvised.
I don’t see why this weakens the example. He has repeatedly built and released games from his own engines and now language/compiler. Money was increasingly required but he had made that money through the same process.
He was able to do it because he had millions of dollars from selling previous games he had written in his own game engines (one of them 3D). If other devs want to try this approach and don’t succeed then so be it, but they’re free to use whatever approach they want. I was simply pointing out that the stereotype presented in the essay has exceptions.
> Ten years ago, the kind of person who implemented a new compiler because their compiler was slow was the kind of person who started programming a new 3d game engine when they wanted to make an indie game, i.e., the kind of person who never ships anything.
Jonathan Blow is a glaring exception to this idea. In order to make his latest game, which has shipped and is great, he wrote not only his own engine but his own programming language and compiler to build it in. Zero use of AI.
Somewhere this analogy breaks down is that the only pair of shoes I own that are completely repairable are the traditionally made leather boots I got for my wedding. But with AI generated code it seems endlessly possible to ask the agents to keep fixing the code as long as you don’t care about code quality or fundamental design changes happening frequently.
EDIT: I guess maybe agentic code maintenance is more akin to just making a new pair of shoes every time the current one shows any deficiency?
You are taking too narrow of a view. I agree that if you want to say something about the behavior of models in use today then you need to use those models. But that’s not the only reason to study questions like this.
Just because AI development moves fast and breaks things continually it doesn’t mean there’s no value in studying older models. Unless you think we understand all the behaviors of those older models.
As others have said it's mostly just consistency of practice. There's a wealth of resources available freely online for these topics and you learn by doing. I would caution setting goals/expectations based on what others have achieved in youtube videos though - this guy was/is a computer science undergrad at Cambridge, so he certainly has a high capacity for learning hard stuff and has been exposed to some of the best schooling on offer.
I think it’s plausible that attention could have been invented earlier and GPT demonstrated at smaller scale on earlier hardware. Then hardware timelines might have accelerated (or alternative approaches explored) and we’d have gotten to where we are earlier.
I don’t understand why this article makes you ask that, can you elaborate? The simple answer to why we didn’t have LLMs in 2005 is that nobody had the requisite ideas that make them work effectively (independent of any agentic harnesses and Schmidhuber notwithstanding) and there wasn’t sufficient clarity in the direction to scale for anyone to build anything at scale (also a requisite for them working well).
I wonder how much bias there is in the science here. I certainly can think better with pen and paper versus computer but I’m 45 and my whole schooling through to postgrad was done with pen, paper and blackboard. It seems very plausible that digital natives from childhood will think just as well at a keyboard (maybe better since they’ll be less practiced with pen and paper).
Right. The person you responded to made a statement entirely about “right now” and used the word “currently”. You added in the discussion about the future, they said nothing about that.
I agree with you about not treating programming as a protected skill that has a right to be well paid, but computers were never “supposed” to be anything. They’re in no way inevitable in their current forms and any ideas about how they were supposed to be were just desires.
My dictionary says that art is “the expression or application of human creative skill and imagination”. That seems like a definition that applies to math.
> But if it does not solve it, will you really wager 30 years of your life on the possibility that you might be smarter than the AI?
Right now it’s not a terrible bet. If the OpenAI results from the past week are representative then the AI success rate for verified proofs on open math problems is somewhere below 10%. There’s a long way to go before you can assume an AI failure to solve means the problem is unsolvable.
> And will conciseness and readability even matter in the long term?
Fair question. Right now the way for humans to understand code is to read it and that'll be easier if the code is concise and readable. But it's possible that we'll end up in a situation where the code is separate from the explanatory materials that humans need. This is similar to math where we might have impenetrable Lean proofs of theorems and then natural language expositions to explain the proof.
> AI agents can build a whole app and review PRs better than a lot of people
Assuming you're comparing to experienced human developers that are good at what they do, then I think you have to define what "better" means here. It's clear that you can get correctly functioning software faster in many cases, but it's not clear that the design will be better in terms of conciseness and readability or that it's as useful as a foundation for building upon later if nobody understands what's going on inside.
Proponents of in-office working and open plan talk about the benefits of spontaneous conversations. For me I find I just can’t get anything done in these environments and I prefer much more deliberate communication. I wonder what the situation is like in tech companies in Japan today?
This. We’ve just moved the bottleneck from during writing the code to reviewing. The failure mode here is that you can choose to just do a light review since the code “works” but if you had to write the code yourself you would have been forced to consider the design choices and broader implications of the code more.
Most helpful way I know to think about this is: if you’d been given that stock as cash would you have bought the stock. Breaks the cognitive bias of already holding the stock.