Well thought out Acceptance Criteria define the boundaries and parameters of a User Story / Feature and allow both the development team and the stakeholder to determine when a story has been completed.
The creation of Acceptance Criteria should be a collaborative approach between the development team and the business, utilising a common language. Too often language used by the business and that used by development teams and found within code differ and lead to confusion/misinterpretation of requirements. Acceptance Criteria can help bridge this divide.
Acceptance Criteria should include both functional and non-functional criteria. Non-functional requirements such as performance, usability and security criteria may not be defined by the stakeholder and as such the initial Acceptance Criteria will be enhanced by the development teams before being signed off by the stakeholder.
The language used during the creation of Acceptance Criteria also becomes important if you are creating automated tests using a BDD tool such as SpecFlow. Not only must you use the Given-When-Then gherkin syntax you also need to consider reusability, carefully wording steps so that the code behind can be reused.
For instance generating the step definitions in SpecFlow for the step "When I click the Submit button" will produce a method like so:
[When(@"I click the Submit button")]
public void WhenIClickTheSubmitButton()
{
//Some code.
}
As clicking buttons is a common step you would then want to make this reusable by replacing the button name with a wildcard, also changing the method name to be more generic:
[When(@"I click the (.*) button")]
public void WhenIClickAButton()
{
//Some code.
}
This method is now reusable, but only if you ensure the Acceptance Criteria uses the same language. This is where reviews should take place to spot issues such as using
"When I click on the Cancel button" rather than "When I click the Cancel button". As it stands those two sentences would generate two different methods (Step Definitions) because one contains the word "on" thus creating two identical tests with the only difference being the wording.
Note: You can combine criteria that are worded slightly differently, but ask yourself if it's truly necessary...
[When(@"I click the (.*) button")]
[When(@"I click on the (.*) button")]
public void WhenIClickAButton()
{
//Some code.
}
Here's a few pieces of guidance when taking a BDD approach with Acceptance Criteria and SpecFlow:
Acceptance criteria should be written in the 1st person
Write it as if the person using the system is stating what happens:
When I click the Search button
Then I see the Search Results screen.
There are four possible components to an acceptance test
These are all mandatory except the 'Given' line:
Scenario (or Scenario Outline): Describes the test
Given: Describes any preconditions that would allow you to start the test from a particular point
When: Describes the action to be tested
Then: Describes the expected outcome
Each of the sections (Given, When, Then) can be enhanced using the keywords And and But.
Given I am on the Search screen
And I have entered "lolcat"into the Search field
When I click the Search button
Then I see search results related to my search
And the first result is “whatever”
But and And are interchangeable in terms of functionality.
Given I have searched for "cats"
But there are no results
…will have the same effect as
Given I have searched for "cats"
And there are no results
There will obviously be times where one makes more sense than the other.
Use consistent screen names
To enable test re-use and to avoid confusion, screens should be referenced in the same way throughout the acceptance criteria and should be specific to the app where necessary. However, you need to check this with your developers, as common test code may have been written which can be reused across different apps.
An Acceptance Criteria dictionary which for instance contains reusable names for commonly-referenced screens and controls is definitely a useful resource for the entire team.
Use consistent wording
To enable test re-use and to avoid confusion, common actions should be worded in the same way. E.g. instead of sometimes writing When I press “next” and other times writing When I click ‘Next’ or When the NEXT button is clicked or When I press the Next button you should pick one and let the team know what you've decided so that a consistent approach can be used throughout the tests (consider adding this to your Acceptance Criteria dictionary).
Know the difference between single quotes and double quotes.
Whenever words are surrounded by double quotes they are interpreted as inputs to the test. For instance:
When I click "Next"
When I click "Back"
Behind the scenes this is just one WhenIClick test, which will have different values ("Back" or "Next") passed into it. However, single quotes are ignored and can therefore be used for emphasis.
Impact Analysis: Always review the relevant acceptance tests that already exist
As with any type of test some may need to be updated, such as when writing acceptance criteria for changes to a screen that already exists (rather than a new screen). If you're developing iteratively you should expect many changes to the software and the tests.
Review the initial draft of the Acceptance Criteria
Developers and Testers should review the criteria as the developer sometimes needs to make small tweaks to the acceptance criteria for code purposes, and the tester may think of scenarios you have not considered. These changes should then be agreed by the stakeholder.
The concept of “or” does not exist.
You cannot say “When I click A or B, Then…”
In fact, this is two tests:
“When I click A, Then…”
“When I click B, Then…”
However, you may well be able to use a Scenario Outline.
Know when to use Scenario Outlines and when to use table parameters
Scenario Outlines allow the same test to be run repeatedly but with different data each time. For instance, you may want to visit the same page several times, and each time select a different item from a dropdown:
Scenario Outline: As a customer I can add differing amounts of the same item to my shopping basket
Given I am on the ‘Product’ screen
When I select <number of items> from the Quantity dropdown for <product>
Then <product> is added to my basket with a quantity of <number of items>
Examples:
| product| number of items |
| cake | 2 |
| game | 1 |
Table Parameters allow several pieces of data to be passed to one test. For example, you may want to visit one page and enter several different pieces of invalid data into the various inputs. These can all be entered into the same page. If you used a Scenario Outline for this, you would end up visiting the same page several times (possibly via a complicated and lengthy flow) when you could get away with visiting it once:
Scenario: If I attempt to pay without completing a required field I am shown a validation error
Given I am on the Payment screen
And I click the Next button without filling in any details
Then I see the following errors
| fieldLabel | errorMessage |
| Card Number | You need to provide a card number |
| Cardholder Name | You need to provide a name |
| Card Security Number | You need to provide a security number |
Multiple Parameters are a lot like table parameters, but are written instead using a colon and a comma-separated list of values enclosed in straight quotes:
Scenario: As a customer I see the Payment Failure screen when payment is unsuccessful
When I visit the Payment Failure screen
Then I see the following options: "Try again", "Edit card details", "Choose different payment option"
Software Development & Testing Blog