1,561 karma · joined February 12, 2014
ajeet@tesults.com
Your situation probably requires a more delicate approach but really it will depend on the culture of the company. Not every company is a massive bureaucracy. There's probably more chance if it's a smaller company that just wants the best tool for the job and deliver results, and if that's your solution that may be fine.
Ive been an Engineering Manager and Head of Quality Engineering and when you end up with hundreds or thousands of test cases, either in one product or service or across many, managing the output of testing becomes a challenge. There may be low priority regressions that you want to fix later and create bugs for and link to failing tests so you know you’re waiting on those fixes before you expect the tests to pass.
If there is a problem with a test itself then assigning it to a team member to fix is useful. Tracking history helps identify flaky test cases - they’re automatically highlighted as being flaky. Consolidating tests from several sources helps understand the overall health of a project.
If you’re maintaining a product or service that goes through a lot of change and there’s say 10 people or more on a team managing all of this becomes increasingly difficult. With respect to pricing it’s free to try and $25/month per target so not particularly expensive.
Someone else made the point about using data effectively and I agree with that too. I’ve been involved with automated testing and tooling for a few years now and solid test reporting and test case management is something I advocate. I’m the lead dev and founder at Tesults (http://www.tesults.com) - try it and push your results data to it and house your manual test cases. There’s a free tier and if you contact me I’m happy to expand it. Basically if you set up your CI system to run your tests regularly, or even if you schedule them to run regularly (E.g. nightly/daily) then having something like this boosts your ROI because you can track history easily, link bugs, and do things like get indications about which tests may be flaky.
We don’t know exactly how small your team is, it may be you have to rely on manual testing if there’s no one to spare but I’d try to move some of your end-to-end to be automated sooner rather later if you end up growing otherwise you’ll make testing a bottleneck to releasing, full regression testing of even a moderately complex application can end up taking days.
To answer the question, I’ve been in this position and it can happen especially when the manager is in a less well paid city/location. Didn’t create tension but did make me realize location/market matters so much more than skillset with respect to compensation.
I have the same issue at Tesults.com. If someone’s never heard of it, I’d describe it as ‘test results reporting’ but yeah it’s a lot more than that, it’s a complete pipeline for handling post testing.
Other general things that I’ve seen help. Code reviews should not be a rubber stamp, the reviewer should understand everything, there should be a degree of overlap of feature development between devs, end-to-end tests to check for regression can be highly useful once a project is if reasonable complexity.
One thing that has been touched upon is that you are setting the culture, if you say you’re hesitant to sacrifice velocity for tests already that will probably become the long term culture and one day there will be no tests and everyone will be afraid to make any changes. Tests do take up front investment but they run multiple times a day and thousands or even tens of thousands of times over the lifetime of a product and pay it back.
I think striking a balance where you’re not overly concerned about coverage but have key end-to-end flows covered can make things more efficient while saving your skin and help avoid unhappy situations. I say limited UI front-end tests (these are the greatest time sink) of key flows, but then as much coverage of apis as you can do.
Also I’m founder at Tesults (https://www.tesults.com). Check it out. There’s a free plan but I’ll go beyond that and give everyone who posted in this thread one full target free if you email me. So you can report your test results nicely too and keep track of failures without cost. Any direct feedback is appreciated.
Not as grandiose and admirable as Space X and Neural Link but in a small way we do make the world a better place, we even have a promise, giving open source developers, small teams getting started, individuals and educators free SaaS to handle something that would otherwise be pain to do on their own anywhere near as well.