Hacker News Reader

Comment by DougWebb | Hacker News Reader

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of issues found post-release.

This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release.

View on HN

What will happen is that devs will ask their QA counterparts to report issues through an undocumented side-channel.

If you give an engineer a career incentive to optimize something, you'd be surprised how seriously some will take it.

Incentives often cause really nasty political wars within orgs.

When a site gives a 500 because a database went down and the web app couldn't connect to it... is that a bug, a missed test case or should ops take a hit on down time? Furthermore, if you argue that the dev team should have reasonable failsafes in code to connect to a db, in a 10 year old organization, should the current Dev team pay for something that could have been in the code for years?

If you set up a system where devs hand off to QA then they hand off to ops to deploy. Furthermore every step is incentivised somewhat against each other. Even if all teams are equal, it ends up in my opinion, to the path to CYA and Waterfall. Everyone is more concerned with problems not being 'their' fault than shipping good code.

You're absolutely right, assuming we're talking about business/consumer apps and not pacemakers or rocket ships.

An incentive/penalty system for bugs without an incentive/penalty system for features completed leads to paralysis. And a complicated incentive system leads to game playing over productivity.

It's an old saying: be careful what you measure because you'll get a lot more of it.

I'm beginning to think that the best incentive system is just a good base pay, with an emphasis on personal learning and growth. Have engineers strive to become better engineers by taking courses, attending meetings and publishing content. Quality will fall out of that as a positive side-effect.

The solution to this is to require a bug/ticket/tracking item as part of the change. Since QA knows that dev needs to create one anyway, they might as well get the credit.

Where this starts to break apart is that QA can get overzealous and file bugs for different incarnations of the same bug.

It's a bit of give and take.

> The top-level QA manager and the top-level Dev manager should both report to the CTO

For an organization with a single product (and, particularly, one small enough that the "top-level" QA and Dev manager are also first level managers), this makes some sense. Otherwise, this means that each product has no common level of technical management below the CTO, meaning it has no meaningful technical ownership.

It doesn't have to be the CTO; as I mentioned in another reply my intent was that QA and Dev need to be equals in the reporting chain, rather than one reporting to the other. That can work at any scale; even on an agile team with one QA and one Dev, they both report to the same boss who's responsible for both sides.

Of you are fine tuning the reporting chain, your org is already hopelessly broken. That is the central planning Communist model.

Have you ever worked for a fortune 100 company? The dev manager has probably never met the CTO. Im in this right now with my currrent job. I still keep my builds and test them longer than i should. Less broken shit gets through, but i regularly delay my qa ba's because of it.

So, not the CTO, but some shared higher-level manager. The point is that the QA lead cannot report into Development, and Development cannot report into QA. Both of those arrangements can lead to a bad actor stopping the other group from doing the right thing. (Eg: if QA reports to Dev, then Dev manager can force bad code to ship, and if Dev reports to QA than QA manager can prevent shipping and force Dev to waste time on excessive/pointless test code development that's not cost-justified.)

I've met people that call something similar 'fake agile'.

Surely everyone should report into the business (end-user on the business side) lead responsible for making this functionality happen. This includes Dev, QA, business. QA should be business? This is no longer 1985, there is no reason business (product owner) shouldn't be a capable PM and BA for their own product.

In the company I used to work for, the CTO and VP of Product Management were equals, both reporting to the CEO. The CTO's reports included the QA director, Dev director, and Project Management director.

On any given project, we had a Product Manager who set the business requirements, a Project Manager who ran the project, a Dev lead in charge of the development team, and a QA lead in charge of QA. None of these people could override the others because of the reporting arrangement, so we had to collaborate instead to make sure we all met our goals.

Early on this arrangement worked really well. But it started falling apart when ownership of the company changed and politics became a factor as people who cared more about moving up through the organization than about the success of the company and quality of the products were hired and promoted. My take is that bad management will destroy any approach you might take to producing good software. That's why I eventually left.

I work for a FTSE 100 company and the dev manager and CTO in the division where I work are very well connected. I don't think the size of company is necessarily the only factor here.

That sounds like a terrible strategy for a lot of us, where time to fix is much more important than bug count.

This is because there is likely to be a tradeoff between speedy development and bug count.

If you are shipping physical CDs of software, or releases which you can't update easily, or if you are working in a critical environment where mistakes are disastrous (finance, health, space, etc.), then it is fair enough to be so concerned about bugs.

But many if not most developers work on the web in areas where the product is uncertain and evolving. Prioritising rate of change is more important here. Rather than extensive QA, monitoring and fast rollback are better choices.

A formal QA process, which goes along with a formal release process and release schedule, certainly does slow things down. I think for large-scale projects with a large feature/requirement count that's ok, because there's a lot of little details to keep track of and the formal process helps with that. But you also need a fast-track for fixing small things quickly, and for that Dev, QA, and Ops need to work together to quickly diagnose, fix, test, and release changes.

My experience with the formal process I've been talking about was in a company whose product is a large-scale web application that interacted with a complex custom back end search engine and a proprietary content database. The QA and Dev teams had a friendly adversarial relationship, but ultimately we all felt we were on the same team with the same goals: to produce the best software we could. It was a fantastic place to work.

The top-level QA manager and the top-level Dev manager should both report to the CTO

Having a top-level manager who is responsible for a whole discipline doesn't work, it just creates a high wall to throw shit over. Developers and QA testers really should report to the same first line manager.

Reply on news.ycombinator.com