Cucumber: BDD Testing and Collaboration Tools
cucumber.io
cucumber.io
- I was an outside consultant working with nontechnical but very process oriented management teams
- There was low trust in existing technical workflows: they had been burned by bugs, failure to deliver, and overpromising so a big part of my task was building rigor and consistency into the engineering workflows
- They were extremely engaged in managing the workflows (the core of the business was doing the thing that we built an app to do)
In that context, they loved having a printed out (!!) copy of all the cucumber tests running on the most recent build so that they could see what was passing and what was failing: Color printer, green line passes, red line fails.
Once I got them into the habit of writing "given when then" statements to define existing features, bugs, and functionality, they were happy as a clam being able to specify things. Eliminated a lot of the usual back and forth over "yes but what exactly do you mean" that non-technical people often create due to not being clear and exact enough for technical purposes.
I would implement the parsing and selenium-based testing that validated whether the cucumber was passing and we would have a shared understanding of where the code was at and whether there were any regressions. In the occasional case where my understanding didn't match what they expected, we had context to work together to figure it out.
All in all it earned me a fair amount of money and lowered the stress of being a consultant for those companies substantially so I have a lingering fondness for it. But like I said it's a very specific situation to deliver that kind of value.
For simple and very clear Given/When/Then scenarios with clear inputs and results that were easily represented with a small amount of text it worked. Think a trading system with a few codes and a small amount of data on the left and an expected result like a calculation on the right.
Where there was a lot of stories that branched off from one another (e.g. a website with lots of potential user flows) it inevitably led to stories that were too vague or too repetitive or more usually, both.
I'm convinced that it does work for a certain kind of application for which the DSL is capable of representing its scenarios, but it's not the kind most of us develop.
There were also a lot of cucumber scenarios and suites involved. Like think dozens of printed pages in relatively small font. Concise it is not. But the people involved wanted verbose and clear over concise and easy :)
It can be a very reassuring way to develop software for stakeholders - "We added 8 scenarios, and made a further 4 failing scenarios pass this week" - as opposed to "our developers spent 6 months and then quit without delivering any code"
I'm glad Cuke made you money. There are lots of ways to make money from mismanaged companies.
If we were actually professional engineers, we'd have a code of ethics. I'd like to think that the ethical code would involve not doing harm to a business or intentionally obscuring harm being done.
I don't care for the implication that I was being unethical - I hold myself to a very high standard, and I do have a code of ethics, thanks very much.
Probably the most important thing was that with a lot of test frameworks, having poorly architected test utility code is annoying and more time-consuming, but has a limited blast radius. Hacky tests can be easily isolated from the other test code. In Cucumber, proper architecture and reusability are paramount. If you screw this up, your tests become unmaintainable really quickly!
- Maintain documented requirements for ALL CODE FUNCTIONALITY - Maintain tests for ALL REQUIREMENTS - Be able to produce reports for auditors that prove linkage between the requirements and the tests (to prove that all requirements were tested)
In this context, Cucumber was really nice since it automatically solved all three of these at once! For general testing purposes, though, I think Cucumber has too much overhead for it to be worth using...
I would say most companies use cucumber as a testing tool and say they doing bdd. I would actually like to see a company doing true bdd I have not seen one.
Didn’t the founder post here and said something along the lines of hating bdd now?
Edit: oh I see cucumber was bought by smartbear and then they fired the last cofounder https://mattwynne.net/new-beginning
Just use a programming language.
[1] https://en.wikipedia.org/wiki/Behavior-driven_development
i know that 99% is badly written sure.
but bdd cucumber style testing if done right can be good. problem i saw people just incapable.
I don’t mind it if you use cucumber and it still serves the purpose of describing your code’s behaviour, so the output is readable. RSpec’s documentation is a nice example of that.
The problem is if people just use it as a syntax over a webdriver, which means that your test behaviour and logic is arbitrarily split into two different languages, the cucumber output is useless, the tests are needlessly slow, and it’s really hard to debug. Cucumber itself is not able to add value there.
But that’s not unique to cucumber either. I don’t think many people realise that test writing and architecture is its own skill in programming and so many suites turn into write-only cludges that take 12 hours to run on a cluster of 50 CI workers.
Cucumber and WebDriver? Two problematic frameworks does not equal a sane one.
Found it quite neutral, not too bad, but also not really helpful.. some nice ideas especially that cross functional collaboration, but in the end the engineers had overhead and hated to write the wrappers, and the other stakeholders still couldn't or didn't want to read/write docs/test, or however it was intended. Admitted, maybe I belong to that incapable crowd as well ;)
Admittedly, I never dealt with Cucumber.
Cucumber mostly exists to upsell naive MBAs and project managers on their "pro" tools.
* It's verbose.
* It's not type safe.
* It desperately needs some kind of inheritance - most user stories branch from other user stories (e.g. put product in shopping cart->purchase product). The only way people who use it can deal with it is by concealing critical details in code or by writing massively repetitive stories.
* What other language has a parser that routinely asks you to write regexes to get things to parse?
None of these things are fixable and the creators have largely reacted to the complaints with gaslighting: "you're just using it wrong".
It can be used effectively if you align the stars correctly but usually not.
The one thing it does get right is not being turing complete - there's clear value in not using code to describe scenarios.
I wouldn't lump RSpec in with the rest of those.
Spec-style frameworks (like granddaddy RSpec) are terser. They don't have layers of indirection. They're intended to help devs think about how code will be used, not to make code look like English.
But it's worth it - I find acceptance tests are great for testing systems from a data perspective, i.e. what comes in to our app, what we send to other apps, what we respond with, etc. The data coming in and out is ultimately what I care about ensuring.
I haven't yet seen acceptance tests that are readable by "business", but they do help remind me what the usecases in our system do.
I can speak for RSpec and say the output of the test runs can be formatted such that it generates specifications that are as "non-technical" as Cucumber. If you scroll down a bit from https://onsi.github.io/ginkgo/#mental-model-how-ginkgo-trave... you'll see a similar output.