General Test Automation Standards
Regression tests must be automated · AT-01 · MUST · TEST
Automate all regression tests to ensure repeatability and reduce manual effort.
Tests must be run automatically when new functionality is released · AT-02 · MUST · TEST
Configure automated tests to run as part of the release process for new functionality.
Tests must be tagged by regression priority and system area · AT-03 · MUST · TEST
Tag each test with its regression priority (such as high, medium, or low) and the system area it covers. This enables targeted execution and reporting. This can be used to run only high priority or relevant tests when new functionality is released
Tests must be atomic and idempotent · AT-04 · MUST · TEST
Design each test to perform its own setup and teardown, including creating and deleting data as needed.
Explanation: Atomic and idempotent tests can run independently and in any order, without relying on the outcome or side effects of other tests. This ensures reliability and repeatability.
Tests must be actively maintained · AT-05 · MUST · TEST
Actively maintain all tests as the system evolves.
This includes:
- Pipelines are checked after each run.
- Any failing tests are addressed as soon as possible following the process laid out in the projects test strategy.
- Reprioritising tests as project priorities change.
- Deleting tests that are no longer relevant or valuable.
Tests must have a descriptive title · AT-06 · MUST · TEST
Give each test a clear and descriptive title that explains what is being tested.
Test titles should include work item ID · AT-07 · SHOULD · TEST
Include the work item ID in the test title to trace tests back to requirements or user stories.
This may be a project-specific decision.
There must be a link between automated test pull requests and work item(s) they test · AT-08 · MUST · TEST
Reference the relevant work item(s) in pull requests containing automated tests to ensure traceability between code changes and requirements.
Tests must have a strong assertion · AT-09 · MUST · TEST
Include a strong assertion in each test to verify the expected outcome.
A strong assertion ensures the test fails if the application does not behave as expected.
Tests must assert against the behaviour in the test title · AT-10 · MUST · TEST
The assertion in each test must directly verify the behaviour described in the test title, ensuring clarity and alignment between test intent and implementation.