Writing Acceptance Criteria (AC)

Acceptance criteria define the conditions that must be met for the software to be accepted by a user or stakeholder.

Each Product Backlog Item (PBI) must have acceptance criteria, which must be signed off by the Product Owner prior to work commencing.

Acceptance criteria should be captured for the benefit/use of all project stakeholders, including end users, the development team, and all other stakeholders.

Criteria should therefore avoid development-related technical terminology and should not be used to record implementation details or the tasks required to develop/test the feature. These can be recorded elsewhere when necessary e.g. in the card description, testing tab, or as tasks.

Using the acceptance criteria, it should be feasible:

• To generate a test case with a clear success or failure outcome. The successful outcome of tests provides the quality assurance that the application works as intended.
• For an end user to utilize the acceptance criteria to define their own test (as part of User Acceptance Testing).
• To confirm what an application does and doesn’t do such that post-development issues can be clearly identified as either a bug (the functionality is not working) or a new feature request (existing functionality is working as originally defined in the acceptance criteria, but our needs have changed).

Styles of acceptance criteria

There are 2 main styles for writing acceptance criteria - rule-oriented and scenario-oriented.

Audacia tends to favour the rule-oriented approach.

In some cases, especially where requirements definition is proving challenging, we use the scenario-oriented approach. The requirements definition process might therefore look like this:

1. Have an open discussion about requirements, without writing much down. Understand what led to this feature being needed, the ultimate aim, and how this benefits the business - so that contextual understanding is good.
2. Draw some wireframes on a board and agree the fundamentals of the user experience.
3. Start writing some acceptance criteria using the rule-oriented style.
4. Get stuck, and try writing criteria using the scenario-oriented style instead.
5. Once a thorough understanding has been gleaned, rewrite criteria using the rule-oriented style, such that it is more concise.

Rule-oriented style

The rule-oriented style simply sets out a number of conditions that can be met, presented as numbered bullet points.

Example:

1. The user will have the ability to set their username in the "Username" field.
    1. The username is required i.e. it cannot be left blank.
    1. The username has a maximum character length of 50 characters.
    1. The username has a minimum character length of 2 characters.
1. There will be a "Submit" button on the form.
    1. If the username is invalid and the user clicks "Submit", they will see an "Invalid password" message.
    1. If the username is valid and the user clicks "Submit", they will see a "Username saved!" message.

Scenario-oriented style (Gherkin syntax)

We often employ the use of the Gherkin syntax.

This defines a “Given, When, Then” structure for writing requirements:

GIVEN [some precondition]
WHEN [I take a specific action]
THEN [I will see this outcome]

“And” statements may be added as required:

GIVEN [some precondition]
    AND [some other precondition]
WHEN [I take a specific action]
THEN [I will see this outcome]
    AND [I will also see this other outcome]

Examples:

GIVEN I am viewing the "Set username" page
    AND I have not already set my username
WHEN I type characters in the "Username" field
THEN I will be prevented from typing more than 50 characters

GIVEN I am viewing the "Set username" page
    AND I have not already set my username
    AND I have not typed any characters in the "Username" field
WHEN I click "Submit"
THEN I will receive a validation error that states "The username cannot be blank"
    AND I will remain on the current page

Note that this syntax is more verbose than the “rule-oriented” approach, but in some cases it encourages evaluation of all logic branches and may make it easier to write test cases.

The INVEST mnemonic

We use the INVEST mnemonic) to confirm that items are:

• Independent - The PBI should be self-contained, in a way that there is no inherent dependency on another PBI.
• Negotiable - PBIs are not explicit contracts and should leave space for discussion.
• Valuable - A PBI must deliver value to the stakeholders.
• Estimable - You must always be able to estimate the size of a PBI.
• Small - PBIs should not be so big as to become impossible to plan/task/prioritize within a level of accuracy.
• Testable - The PBI or its related description must provide the necessary information to make test development possible.

The following rules should be followed when writing acceptance criteria. These will be split into 3 categories; Must, Should and Avoid. They are categorised under the AC ruleset for easy reference.

Use Numbers · AC-01 · MUST

Using enumerated AC must be done as it enables easy reference to a specific point.

This can include sub numbers. For example, 1.a.ii.

Do this

1. New navigation menu item for Users

Don’t do this

• New navigation menu item for Users

Avoid Implementation Details · AC-02 · MUST

Criteria should be written from the perspective of the end user who gets value from the feature.

In most cases, the developer of a work item will be free to implement the feature as they see fit, so technical implementation details will generally be avoided in the acceptance criteria.

Do this

Given I am logged in as an administrator, when I go to the user management page, I can add a user

Don’t do this

3. Add a new class called AddUserService in the Services/Users folder

There may be cases where this is unavoidable. For example, an implementation detail for which a decision has already been made might align with a specific element in the acceptance criteria. In these cases, this aspect should be clearly marked. An example:

1. Given I am viewing the documents form, when I click then ”+” button, then I can upload a photo.
   1. Developer note: all documents will be stored in Azure blob storage. This aspect does not require QA/UAT testing.

Here it is made clear who the intended audience for the note is, and whether this aspect is testable.

NOTE: This rule can be broken when a card is purely technical in nature, for example when resolving technical debt.

Define Negative Outcomes · AC-03 · SHOULD

As well as defining what a work item does do, AC should also include negative outcomes where necessary to provide clarity.

Do this

i. When an invalid email address is entered, display an error message stating that this is the case.

Exclusions · AC-04 · SHOULD

Where necessary, include details of any work that will be explicitly excluded.

Do this

5. This does not include the setting of the new user’s password (this will be covered as part of a later PBI #XXXXX).

Avoid Making Assumptions · AC-05 · MUST

Assumptions and ambiguity should be avoided. It should be possible for someone with limited system knowledge to understand the criteria.

Do this

3. This functionality should only be accessible to users with the “Identity Manager” permission.

Don’t do this

3. This functionality should be restricted as usual.

Concise and Structured · AC-06 · MUST

AC must be written concisely but in a structured way such that points aren’t easily missed. This might mean separating one point into multiple.

This leads to AC points that are easier to verify.

Do this

1. Upon entering an incorrect username/password a message should be displayed that
    a. States the details provided were incorrect
    b. Asks the user to try again
    c. Advises them to use the forgot password functionality if still not right

Don’t do this

1. When a user enters an incorrect username/password combination then a message should be displayed to inform them of this and that they need to try again or use the forgot password functionality.

Miscellaneous Considerations · AC-07

Performance · AC-07.1 · MUST

If there are any particular performance targets that need to be met either now or at a point in the future when the system is under a greater load, this should be captured.

Security · AC-07.2 · MUST

If there are any particular security considerations for a piece of functionality, these should be included.

For example, if there is a file upload, should they be checked for viruses?

Permissions · AC-07.3 · MUST

What permissions are needed to be able to use the functionality being defined?

Not Testable · AC-07.4 · SHOULD

If an element of the Acceptance Criteria is going to be impossible to test effectively, this should be recorded for the benefit of QA/UAT testers.

Workflow · AC-07.4 · SHOULD

If appropriate, provide a path through the system to get to this new feature e.g.

1. When the user navigates to Admin Menu > Users > Add User, they will see the “Add User” form.

Validation · AC-07.5 · MUST

Validation requirements must be defined where appropriate. If we’ve defined an input field:

• As a text input
  • Does it have a minimum/maximum length?
  • Are there any constraints e.g. “cannot contain hyphens or asterisks”?
  • Can it be whitespace only e.g. ” ”?
• As a number input
  • Does it have a minimum/maximum value?
  • How many decimal places can the number be specified to?
• Is it optional or mandatory?

User Experience and Aesthetics · AC-07.6 · SHOULD

Are there any specific requirements in terms of the layout, colour scheme, amount of visible information on page etc.

Enduring · AC-07.7 · SHOULD

Is the functionality as defined enduring i.e. will this make sense in 6 months, 1 year or 2 years time?

Consistent Domain Terminology · AC-07.8 · MUST

Be very consistent when using critical domain language.

For example if we refer to “System Administrator” in one place, we should use this consistently everywhere – rather than using “Admin” in one place and “System Administrator” elsewhere.