> i tried running npm test and it was completely borked
> one problem, of course, was that the tests were entirely bullshit.
> whenever it would start getting lazy or confused i'd restart the session. often a failure would "demoralize" it or being sloppy once would cause sloppiness to stick. in particular i've noticed that being overwhelmed causes it to approach problems in a messy "throw anything against the wall" way. sometimes if too many newly un-skipped tests are causing failures and it got "demoralized", i'd just skip them again and have it focus on one or two at a time. with less noisy output and a permission to "really dig into what happened" (and often an explicit suggestion to remove things from the example until it no longer breaks), it would usually find the root cause.
> we got to majority of passing tests but there were a bunch of bugs it just couldn't solve and would walk in circles. the code was also getting quite complicated. it seemed like a mess of different ideas and special cases thrown in. moreover, i knew it didn't fully work because i had new test cases that just would refuse to pass
> i tried to let it do that and it just failed miserably anyway, breaking tests and not being able to recover.
> it struggled at first but i reminded it to look at other emitters.
> at one point, newly added tests kept confusing claude. it would completely get stuck on them, failure after failure, fixing one thing and breaking other thing, trying to turn off those tests or change the expected outputs (despite me telling it to never do that!) and in general seeming aimless and distraught (in the descriptive sense).
> i had to git reset --hard multiple times in this mess.
—
tldr; seems like a very nice experience, yeah