ADAdam_Evans·Sep 10, 2015
I'll take the bait.
Cucumber is a testing tool, depending how you use it it may facilitate BDD. It may not be your intention as the author for it to be a testing tool but it is, lets look at it some more.
Gherkin (Given-When-The)
Scenario: Close taxis with higher rating win
Given taxi A with rating 0.8 is 1400m from the customer
And taxi B with 0.9 is 1500m from the customer
When the customer requests a taxi
Then taxi B should be assigned
Lets look at the Gherkin syntax. It follows a set format with forced English language. It gives a context (Given), some input data (When) and an expectation (Then). This syntax is an example based specification and lives in a plain text file generally with the file extension *.feature. We'll call the above example a feature based on the file extension, we'll also call this Gherkin format
Read the rest
an external DSL (domain specific language).
Now on it's own this feature file is useless, it's a plain text file. Why do I need Cucumber for this it's a plain text file? Why can't I store it in a shared wiki where people can collaborative edit it and track changes? If it's a plain text file to share knowledge why do I need to use forced English with the Gherkin syntax instead of a more natural form for the intended audience? Why does it need to be text if it's demonstrating shared knowledge, maybe a comic strip may be more appropriate for the domain? Why would I need Cucumber for a team to sit down to create this collaborative knowledge?
We need to follow this strict syntax because it's an external DSL which is used by an interpreter. This interpreter parses a feature file and asserts a given input is equal to the expected output specified in the feature file Gherkin DSL. Lets think about this some more, we give some context, we provide some data, we run a computation and we assert the output, this sounds very much like an automated test. If I was to write a definition of an automated test this would be it.
This interpreter is fairly fragile. We have a miss match between the plain text feature files in our Gherkin DSL format and our test (not sure what else to call it?) execution tightly coupled with fragile regex. The implementation also promotes heavy mutation in the test (sorry, again it's not a test?) implementation which ultimately leads to fragile assertions (aka tests).
I know you say don't use UI testing tools with Cucumber, but I'll take your own example as it demonstrates mutation quite clearly https://github.com/cucumber/cucumber-jvm/tree/master/example...
The steps are stand alone
@Given("^I am on the front page$") will mutate sending the browser to a page. It doesn't give an indication if it's successful or not. The function is marked with throws InterruptedException so I can only guess it blows up if it fails, or maybe not?
Then we have in TemperatureStepdefs @When("^I enter (.+) (celcius|fahrenheit)$") which finds an element by id and sends some key events. Again how does the subsequent step know this is successful, how does this step know the previous step was successful? We don't, we assume, our test may work or we may get silent errors which cascade down.
Then you see people use things like public String currentPage = "" and update it as you progress through the workflow and assert on it. Mutation, race conditions, silent failures if you've used Cucumber for any amount of time you've been deep in these trenches.
I digress
So, if Cucumber is not about the test part why do we need the interpreter which runs a computation with given input and tests it matches the expected output. Without this part Cucumber is a set of flat text files. What do we get? flat files? Why are these better than a shared wiki, google doc, spreadsheet? They achieve the same, canonical source of knowledge.
But Cucumber promotes a conversation. No, used in an agile productive organization features will be written collaboratively and Cucumber may be a facilitator to reach this goal. Cucumber does not enforce this or ensures this happens, it's promoted but in no way is this a requirement to use Cucumber.
If you are already an efficient team delivering software you'll already be having this communication part. Cucumber isn't a tool for test so what does it give us if we are already talking and delivering? We have other better test tools (cucumber is not a test tool right?) and more efficient ways of collaboratively writing, sharing and tracking knowledge.
Cucumber is a facilitator to help organisations to start having conversations and collaboratively share knowledge. I accept this, people who are looking at ways to improve things see this as a valid usecase, it's a trojan horse to get a more agile workflow in through the backdoor. My issue is when Cucumber is used in this way it's used when you are knee deep in mud, it's a technical solution to mostly a non technical issue. Your organisation is dysfunctional, likely not delivering and a command and control structure. Your team looking at Cucumber want change but the issues lie far deeper and Cucumber will not save you. Open up a Google Doc, Wiki Page, sit down with your team and stakeholders and first talk.
So really, what is Cucumber?
As a test tool it sucks. There far better automated test tools
As a shared knowledge base, it's easier to collaborate in something accessible to everyone where change is easier and can be audited. A wiki, a Google Doc, a spreadsheet, it really doesn't matter they all achieve the same goal.
As a facilitator, you have a non technical problem deeply rooted in how your organisation works. Sit down, have the talks, create the shared knowledgebase, solve the core issues then and only then look at technical facilitators.