The Siren Call of Automated Browser Testing
cugablog.wordpress.com
cugablog.wordpress.com
This is the reason I follow this pattern for automated testing - only use readable selector IDs (for individual elements) and only use readable CSS selectors (for items in a group):
https://hitchtest.readthedocs.org/en/latest/faq/why_just_htm...
This ends up leading to tests that look a bit like this:
- Click: view-button
- Verify text:
item: first first_name
text: Sanders
I've tried most other methods of automated testing too (selenium IDE, page object), and this one is the most straightforward and sensible IMO.
I would probably use this for BDD + ATDD too although I've yet to work on a project where that's been possible.
>The time it takes to run the tests is often exacerbated by a lot of explicit delay statements in the test cases.
This is an antipattern. You shouldn't do it:
https://hitchtest.readthedocs.org/en/latest/glossary/sleep_o...
Unfortunately, avoiding in most frameworks requires the tester to write asynchronous multithreaded code (which is beyond most QA developers).
IF your browser test isn't written in high-level concepts (page objects at the very least), and if your developer making UI changes isn't getting immediate feedback to also update the library of test code he's affecting, yes, it'll be fragile and will suck up too much developer effort. (I'll stop short of asserting the opinion that the UI developer should be changing the tests' reflections of his UI components BEFORE making the changes themselves, but I don't think that position is entirely unreasonable)
It should not be hard to find a balance of engineering practice (reuse, modularity, etc.) that gets you a degree of maintainability that is economical (utility provided by the system exceeds that consumed) -- just like any other kind of code.
Anytime someone messes with an interface, they should have roughly one place to adjust things for their changes. Dashboard controller and views get completely rewritten app side? Dashboard page object (maybe) gets rewritten test side.
Things should be written in the first place for maintainability -- XPaths/selectors chosen carefully, using user-facing labels if possible, WITH collaboration from the UI side -- no stupid duct-tape shit because you don't have a good way to get at something (like "the fourth input following the second instance of the word 'Account'").
Like any kind of code subject to change, you must invest in engineering for robustness in the face of that change.
No, you can't have a mandate from pointy-hair types to pour unskilled, unplanned effort into a system or practice because it sounds nice and expect to break even, but no shit -- that has nothing at all to do with browser automation.
Depending on the economics of the team, product, even the code-illiterate guys running the selenium recorder in another timezone and sending in bug reports any time it complains have their place (for instance, in a year-long release cycle of mostly stable line of business software where there are few UI changes because they'd require big change-orders and rewriting all the documentation) -- you have to design your strategy for your product.
QA and Product, too. A really good QA is the best person to write user stories with a customer because they have a natural ability to sniff out edge cases which they can then walk the customer through.
However, the issues remain the same as with any other ABT test. An async element might take a while to load due to latency, and isn't there by the time your test says to click a button, or browsers were updated and now render your CSS just differently enough to break the test.
Perhaps part of the challenge is there are so many things that can change and break the tests outside of the actual code.
> Thinking realistically about automated browser testing.
But great article none the less.