boredtofears

Do LLM's not benefit from the abstractions higher level languages like Javascript/node offer?

Perhaps I'm speaking out of depth because I haven't done a lot of Golang, but I've always thought of it as a systems language first, which means by necessity you have you to handle lower level problems yourself. I'm sure there's plenty of libraries that paper over this - but the philosophy of the languages themselves is different. Javascript was designed to solve CRUD like interfaces/problems quite well.

Maybe this is just an outdated argument though that isn't really relevant with modern golang/rust though.

All of those things are smells imo, you should be very weary of any code output from a task that causes that much thrashing to occur. In most cases it’s better to rewind or reset and adapt your prompt to avoid the looping (which usually means a more narrowly defined scope)

Its a lot less odd when you remember that he's running an AI company himself.

Are you telling me that my home assistant enabled humidity sensors in my garden that trigger the arduino hose valve could just be replaced by a watering can??

It’s not X it’s Y is one of the most obvious LLM writing patterns. Especially the heavily punctuated sentence structure.

I'm on FB primarily because my local buy-nothing group is on it, so I am logging in multiple times a day. I'm so used to this slop it's pretty funny at this point, but as is the case with all social media, you tune your algorithm as you engage. At this point it pushes things like cooking videos and hockey clips more than the AI slop for me.

Sometimes I'll go down a rabbit hole of clicking AI generated videos just because my curiosity is piqued, and then I'll be stuck getting that slop fed to me for the next week. I have to make a mental note to actively disengage with it as quickly as possible to tip the algo in the other direction.

I guess you're right - getting your podcast big enough that it becomes a necessary checkbox for book/media tours is a skill. You're correct that he brings absolutely nothing to the podcast, but he interrupts plenty - usually with superficial pet theories about the "oneness of the universe" or "how all we need is love, actually". He never seems well prepared for his guest beyond a chatgpt summary, never gets any kind of interesting answer out of a guest that they weren't already going to give, just absolutely zero criticality to anything in the interview.

A podcast with guests is an interview. Interviewing is a skill. The difference between a good and bad interviewer is night and day.

I can't think of an interviewer who interjects their viewpoint more and tries to get his guest to acknowledge/agree to his typically shallow level analysis than Lex. The only redeeming quality about his podcast are the guests he gets. I don't think Dwarkesh is great but he's leagues better.

Helping out with a freelance project I built 15 years ago. It didn’t end on the best of terms, but the relationship has since been repaired (and I’m much better at managing my time now)

It’s been fun to come back to, most of the code I wrote still drives the business (it’s just far outdated).

I was pretty early on in my career when I wrote it, so seeing my mistakes and all the potential areas to improve has been very interesting. It’s like buying back your old high school Camaro that you used to wrench on.

What problem is it even solving? Keeping my car straight so I can be less attentive on the road?

I get it in the context of driverless but find it nothing but annoying as a driver.

Of course I use code generation. Why would that be mutually exclusive from AI usage? Claude will take full advantage of it with proper instruction.

As most are

In my experience AI is pretty good at performance optimizations as long as you know what to ask for.

Can't speak to firmware code or complex cryptography but my hunch is if it's in it's training dataset and you know enough to guide it, it's generally pretty useful.

I dunno, when you've made about 10,000 clay pots its kinda nice to skip to the end result, you're probably not going to learn a ton with clay pot #10,001. You can probably come up with some pretty interesting ideas for what you want the end result to look like from the onset.

I find myself being able to reach for the things that my normal pragmatist code monkey self would consider out of scope - these are often not user facing things at all but things that absolutely improve code maintenance, scalability, testing/testability, or reduce side effects.

If you're using a code coverage tool to identify the branches its hitting in the code, you at least have a guarantee that it is testing the code its writing tests for as long as you check the assertions. I could be codifying bugs with tests and probably did (but they were already there anyways). For the purpose of upgrading OS libraries and surrounding software, this is a good approach - I can incrementally upgrade the software, run all the tests, and see if anything falls over.

I'm not having AI write tests for life-or-death software nor did I claim that AI wrote tests that all pass without updating any code.

Also funny because there are many product categories on amazon where if its not above 4.5 its probably shit

I used a static analysis code coverage tool to guarantee it was checking the logic, but I did not verify the logic checking myself. The biggest risk is that I have no way of knowing that I codified actual bugs with tests, but if that's true those bugs were already there anyways.

I'd say for what I'm trying to do - which is upgrade a very old version of PHP to something that is supported, this is completely acceptable. These are basically acting as smoke tests.

You should read my other comment - I did check that the test was actually checking the logic, so I guess I did some level of review with it.

To be honest, I didn't really think about the goals of the projects at all. I'm not even sure if a CONTRIBUTING.md existed. I just needed a particular feature for something I was working and it felt like "being a good OSS citizen" to at least offer it back to the maintainer, but I think I just ended up putting them in a position where they felt like they had to make it work.

Yes, the first hour or so was spent fidgeting with test creation. It started out doing it's usual whacky behavior like checking the existence of a method and calling that a "pass", creating a mock object that mocked the return result of the logic it was supposed to be testing, and (my favorite) copying the logic out of the code and putting it directly into the test. Lots of course correction, but once I had one well written test that I had fully proofed myself I just provided it that test as an example and it did a pretty good job following those patterns for the remainder. I still sniffed out all the output for LLM whackiness though. Using a code coverage tool also helps a lot.

Here's an example:

I recently inherited an over decade old web project full of EOL'd libraries and OS packages that desperately needed to be modernized.

Within 3 hours I had a working test suite with 80% code coverage on core business functionality (~300 tests). Now - maybe the tests aren't the best designs given there is no way I could review that many tests in 3 hours, but I know empirically that they cover a majority of the code of the core logic. We can now incrementally upgrade the project and have at least some kind of basic check along the way.

There's no way I could have pieced together as large of a working test suite using tech of that era in even double that time.

That sounds more like confirmation that greptile is being included in a lot agentic coding loops than anything

Yeah, I kinda stopped involving myself in other people's OSS projects a while ago for that reason. If I have an itch to scratch, I just use my fork. It usually feels like my itch isn't theirs and I always feel like I'm imposing on the maintainer's vision or at best just taking time away from them. I think maintainers have a lot of pressure to accept things because "open source!".

Reading the various comments in the CircleMUD source is a bit of a treat: https://github.com/Yuffster/CircleMUD

Did you get anything workable out of it? How much money did you end up burning? (If you don't mind me asking)

This article makes a lot of definitive claims about capabilities of different models that don't align with my experience with them. Its hard to take any claim serious without completely understanding the state of the context when the behavior was observed. I don't think its useful to extrapolate a single observation into generalized knowledge about a particular model.

Can't wait until we have useful heuristics for comparing LLM's. This is a problem that comes up constantly (especially in HN comments...)

These anecdotes feel so worthless. I notice almost no difference between the two and get generally high quality results from either. This is also a worthless anecdote. I'm guessing what kind of codebase you are working in matters a lot as well as the tasks you're giving it.

Care to share the prompt? Sounds useful!

I've spent a heck of a lot more time grinding leetcode than I have working on take-home projects. I always enjoyed doing take-home's because I could really spend time on it and make it something worth showing off - if anything it always felt like the perfect low-stress way to show what you can do. It's amazing how many candidates don't take the time to make it look good (or even meet the objectives in many cases).

Haven't done one since pre-LLM era though and that path seems like it might be completely infeasible for employers now.

That said, the most productive interviews I've been a part of as both employee and employer have always been with the technical people that you'll actually work with and conversational in nature. You can learn a lot about what someone knows by listening to their experiences and opinions (but this depends greatly on the quality of the interviewer)

It would be nice to see an actual example of what a good claude.md that implements all of these recommendations looks like.