Shouldn't the developers be falling over themselves to fix all those flaky unit tests? I can't believe Google just lets them keep running unfixed. In my code, I don't check anything in unless all unit tests pass (without flaking).
They’re hard to fix. If you can’t reliably reproduce then you don’t even know if you that it’s fixed or you’re just lucky.
They’re hard to find. The change that they first appear tends not to be related to the cause. Leads to a lot of “not my problem”
Really that’s just lack of ownership. Ultimately who owns the entire codebase. You say “my code” but I see a lot of people thinking that means the files I edit, not everything
someone has to explicitly decide to spend day(s) fixing tests, and then only might fix a couple. The roi is super frustrating and it’s hard to spend whole days on a single test.
There’s a ton of broken window theory. Once you get used to hitting rebuild, you just always hit rebuild and don’t even know it’s a new flake.
The most effective thing I’ve seen when you hit this point is get all the test data, junit xml say, somewhere queryable and make graphs and explicit parts of okr with time against it.
The worst was that some tests were actually wrong and only passed due to nondeterminism. Once I made the tests deterministic, I discovered they always failed. Until that point, I found those test failures very confusing, because it seemed like my efforts to improve the reliability of the tests was having no effect on them.
* The test cases written by the some guy who left the company are now your test cases.
* Before he left the company, he was able to commit the test cases, passing whatever review / quality processes were put in place by the company (or that the company failed to put in place).
* If your team wants to have the benefit of reliable testcases, the knot needs to be untangled. That doesn't entail "dropping everything" (although perhaps thats the only way your organization knows how to operate), there is no reason that improving test cases can't be prioritized alongside and in the context of other work.
Unfortunately, in my experience it's usually the last bullet point where things fall down. Product owners consistently pressure development teams to deliver features now, and do test cases later, which means never.
The test dealt with a "week", and on Saturday evening, the current week in UTC differed from that of the local timezone.
I don’t know why the developers don’t care but I suspect they’ve learned to ignore specific tests which repeatedly fail.
Like in most development places management doesn’t give a flying shit about all of the tests passing if it means a feature will ship.
Probably they’re mostly older tests and so fixing them detracts from adding a new fancy feature