Hacker News Reader

Comment by malfist | Hacker News Reader

> I looked at a pre-AI project I did a couple of years ago: test to code LOC was 0.15:1. The projects I work on now are around 1.3:1.

Lines of code is a poor metric for measuring, both for application code and testing code

View on HN

While in general that is true, this is one person and so we can assume a lot of the variables that make it a poor metric are constant in this case and so we can draw a few conclusions.

From my own experience, LLM generated code having more test cases is neither surprising nor an indication of improved quality. I've found LLMs are quite good at making lots of test but the tests are often not that useful. I would argue that total count remains a poor metric for quality.

I don't know of a good metric for quality. Total count of tests is the best I've found, but I agree it is bad.

Best metric is user satisfaction. Since that is ultimately what software is (or should be) about. Hence the leading header of "software is getting worse for obvious reason": we detached from user satisfaction as the metric to optimize for at least a decade ago.

Ask for red-green tests. Without that they are often indeed useless.

I'm curious, mind to tell me more about your process?

Just simply ask for that, and they usually know what to do.

They should write a test and stash the fix they just wrote. In this state it should fail. Once they re-apply the patch, it should become green and this gave use enough information to deduce that the test does test something.

We can conclude that there are more tests per code perhaps. We can't conclude they're adding value.

my experience is 25% of valuable. Which is a lot better than nothing.

I think you meant it's a poor metric for judging. It's fine for measuring. Or do you think git should stop showing +/- line counts for PR's.

Somehow LOC morphed from one metric among many to "you should never even think about LOC, 15 lines and 15k lines are exactly the same thing". It's weird.

Reply on news.ycombinator.com