Hacker News Reader

Comment by ipsod | Hacker News Reader

Can you not seek mastery with the new tools, though?

It does feel a bit like building on quicksand, but coding has always been that way for me. Well, less so with embedded than webdev, but still.

View on HN

I used to find joy in knowing how things worked inside-out. Whether it was Postgres, or browser engines, or some detail about compilers... I used to find joy in developing the little techniques that made programming NOT building on quicksand. There was satisfation in what you had done being robust.

Pardon my lack of creativity but I'm not even sure what mastery with the new tools (LLMs) would look like. When you are so abstracted from the details and don't have to know anything, what _is_ mastery? (I am not asking rhetorically and if people have overcome this thought I would be interested in hearing from them.)

Knowing how to ask a machine questions in the right way? Playing as a manager in control of non-human developers? It all sounds really silly and something that is difficult to aspire to (and I can't imagine explaining my job to others with a straight face). I think I find it even more egregious because it's all built on paying rent to the LLM operators.

It's not that I don't enjoy results in their own right sometimes but I think I am definitely in the 'this is giving me an identity crisis' camp.

How do LLMs work? Yes there's a model and there's inference, but dig deeper. How can you, today, with < $50, train a small GPT-2 class model, how do you fine tune it, what training data do you give it, how do you give it training data, what algorithms do you use for reinforcement learning, what even is RLHF? You don't have to know any of those details to use ChatGPT, just like you don't need to know how a D flip-flop works in a CPU to write some code that runs on one, but if you're looking for feeling like you know what's going on, at their heart, LLMs are still next word prediction engines, so knowing, deeply, how that engine is created and works, would be my idea of mastery of it. Karpathy has a good set of tutorials on this.

>How do LLMs work? Yes there's a model and there's inference, but dig deeper. How can you, today, with < $50, train a small GPT-2 class model, how do you fine tune it, what training data do you give it, how do you give it training data, what algorithms do you use for reinforcement learning, what even is RLHF? You don't have to know any of those details to use ChatGPT, just like you don't need to know how a D flip-flop works in a CPU to write some code that runs on one, but if you're looking for feeling like you know what's going on, at their heart, LLMs are still next word prediction engines, so knowing, deeply, how that engine is created and works, would be my idea of mastery of it. Karpathy has a good set of tutorials on this.

Knowing how an LLM is built, is trained, inferences, isn't "mastery" in the sense that crafts people use the word.

Mastery to a craftsmen is as much a product of mechanical performance as it is knowledge. For a software craftsmen, that means knowing how to code, debug, and architect a software application by hand.

This, plus since you can't realistically run these LLMs at home, the motivation to get invested in them is not really there (for me at least)

I mean, yes I do find the mathematical background of LLMs fascinating, but it's not going to be something that I can put to practical use (much), if that makes sense?

When I would code by hand I often found I was concerned with the house of cards in my head, am I remembering the right variable names, did I forget something that would lead to an exploit, etc. Now I can direct the LLM to do the work and the skill is split between having an understanding of the interactions of the technologies, choosing tech that will be less of a pain to update/migrate. You can even put some LLM-bow grease into finding more efficient approaches catching corner cases etc. And just by asking I can have test frameworks setup, I can drill into some pet peeve and it takes less time. Now we still have lots of ops moments, like the radicel distributed git forge forgetting to use encryption for private repos, and I run into this class of problem in my own casual LLM use, but that's where our taste and attention to detail powers need to be spent in this modern developer world.

Thanks for your perspective

I think one thing that troubles me is that I so far have gotten quite far by 'caring about the little things'.

Now we live in a world where this feels, for the most part, unsustainable to keep up with. When directing LLMs to code something up, it's still occasional useful to have the knowledge and experience of those little things earned over years, but at the same time 1) it seems like I run into fewer opportunities to add to that experience and 2) it seems like you can often just prompt LLMs to 'grill' a design a few times, it will probably surface some of the most important tradeoffs

Yes, the grilling part is where the LLM energy drain is for me, I haven't released any of the software I've been working on yet because they lack polish in my opinion. They may have nobel goals, but no point unleashing more slop on the world. Often when I ask for a correction something new breaks, or it reveals the next corner the model and I didn't think about. I've seen the models come a long way in a short time. I'm tinkering and paying by the token for deepseek flash picked for its economy and using it to teach myself and form opinions on languages and frameworks that are new to me. It's hard to describe the worth of these opinions, but I think they're important, kinda like why I read a lot of HN looking for morsels of sentiment around the tech industry as represented here.

It's funny though, llms are quite terrible at most things really.

They can build things, but it's all terrible really. Their "ideas" are terrible.

Well, I'm somewhat minded to agree with you, but they are fast enough, cheap enough and don't get bored of tedious work, which makes them passable at many things. Maybe.

I feel caught between two worlds, I know people both very 'for' and very 'against'. The business is very keen for people to use them to go faster.

For non-trivial changes, output so far does not meet my quality standard and I tend to spend a lot of time cleaning up after them, probably more than I would have spent doing the work myself. This being much more boring, I hasten to add.

Their comments suck so I feel like I have to go in and rewrite them all.

Maybe I should just let go of the quality standard and accept that in exchange for steaming ahead at full speed, but I'm reluctant (and not just for me, there are also coworkers who I feel like I would be letting down).

I've always been either entirely self-directed, choosing my own goals, or almost entirely self-directed, being given high-level goals, and getting to choose my own means to accomplish them. So my opinions are probably different from somebody who was a smaller part in a chain.

You say that you liked learning how things worked inside out, whether it was Postgres or whatever. You can still do that. You can learn it at a high level, and then it becomes like a magic word in your arsenal. You don't have to know or remember the details of implementation, but only the nature of its costs, benefits, and synergies with other systems. You just have to remember that high-level abstraction, maybe just a single vocabulary word, of a low-level reality - the exact syntax is really no longer a concern.

The better the agents get, the less there is playing manager. Really your job now is to have and execute vision. Execution is largely a function of having a map of the high level vision and low-level realities in your head, or just having in-the-moment intuition about these things... then just speaking it.

Also, there is developing personal libraries. You may find that in your business you repeatedly tackle similar problems. You develop and refine a library that your future agents can utilize. You develop prompts that make your work more automatic and more robust. And when I say you, I mean, you direct your agents to do it.

It's not so different from what it always was, except that years get compressed into months. What was slow and tedious is now... less so.

I don't know how much advantage I have in this new world, having programmed (sometimes more, sometimes less) for 30 years. My brother has never written a line of code in his life. He's been heavy into AI for a while, and he has developed some really incredible stuff. I think he may have already outpaced me. His $1500/mo+ in subscriptions definitely help.

Did a tailor gain mastery of the sweatshop machine? Did mastery of the automated sewing machines give the same joy to the tailor or was it mind numbing meaningless activity that he would be happier to be free of.

> It does feel a bit like building on quicksand

that's a good analogy and puts into words the way i've always felt about software development. Every so often the entire industry rolls over into something new. I've been here 30 years now and, in my experience, the skill that counts is being able learn new skills. To go back to the analogy, to build something new on top of what you've built in the past before everything is swallowed by the quicksand.

Reply on news.ycombinator.com