Hacker News Reader

Comment by teiferer | Hacker News Reader

> I automate nearly all my tests with AI

How exactly? Do you tell the agent "please write a test for this" or do you also feed it some form of spec to describe what the tested thing is expected to do? And do these tests ever fail?

Asking because the first option essentially just sets the bugs in stone.

Wouldn't it make sense to do it the other way around? You write the test, let the AI generate the code? The test essentially represents the spec and if the AI produces sth which passes all your tests but is still not what you want, then you have a test hole.

View on HN

I'm not saying my approach is correct, keep that in mind.

I care more about the code than the tests. Tests are verification of my work. And yes, there is a risk of AI "navigating around" bugs, but I found that a lot of the time AI will actually spot a bug and suggest a fix. I also review each line to look for improvements.

Edit: to answer your question, I will typically ask it to test a specific test case or few test cases. Very rarely will I ask it to "add tests everywhere". Yes, these tests frequently fail and the agent will fix on 2nd+ iteration after it runs the tests.

One more thing to add is that a lot of the time agent will add a "dummy" test. I don't really accept those for coverage's sake.

Thanks for your responses!

A follow-up:

> I care more about the code than the tests.

Why is that? Your (product) code has tests. Your test (code) doesn't. So I often find that I need to pay at least as much attention to my tests to ensure quality.

I think you are correct in your assessment. Both are important. If you're gonna have garbage code tests, you're gonna have garbage quality.

I find tests easier to write. Your function(s) may be hundred lines long, but the test is usually setup, run, assert.

I don't have much experience beyond writing unit/integration tests, but individual test cases seem to be simpler than the code they test (linear, no branches).

Reply on news.ycombinator.com