Best Practices for Acceptance Criteria and Acceptance Tests
Acceptance Criteria and Acceptance Tests
Sometimes it can feel like the Wild West when it comes to IT and business terminology! Frameworks upon frameworks exist at varying levels of specificity trying to help IT and business professionals not only make sense of their world, but do so collaboratively. Each framework or methodology – whether it is from the realm of Agile, project management, product management, quality engineering, or other – has a set of often-overlapping terms that can be used in different ways depending on the framework or context.
People favor the terms tied to the frameworks or methodologies they have experience with, akin to putting a whole bunch of IT and business terms into a giant bowl of alphabet soup. When we gather around that alphabet soup, peer into the bowl, and dip our spoons in to take a taste, we may think we scooped up the same sets of noodle letters and experienced the same flavor, but often we are talking about slightly different experiences.
We have written in the past about decoding buzzwords around test automation. In this post, we are going to attempt to cut through some of the confusion around the term acceptance criteria and its dear friend, acceptance test.
The term “Acceptance criteria” lives in that alphabet soup, and ironically, it is used to cut through that alphabet soup and facilitate collaborative agreement. Let’s dive in, shall we?
What are Acceptance Criteria, and how do they differ from Acceptance Tests?
Acceptance criteria are, at the root, “conditions of satisfaction” that define when the deliverable or work is complete and has been accepted (Mike Cohn, 2013). They are also sometimes referred to as requirements.
Acceptance criteria define the “what.”
Acceptance tests are the actual, structured tests of acceptance criteria. Depending on the granularity level of a set of acceptance criteria, a given criterion could require a few test scenarios or “acceptance tests” to validate it has been met.
Acceptance tests define the “how.”
When an acceptance test passes, if all parties agree to use the acceptance criteria and acceptance test in the intended manner, it indicates that the deliverable or work is complete to stated requirements and has been accepted.
Here is a simple example of acceptance criteria in Given/When/Then format:
And an acceptance test of those criteria:
Are all acceptance tests User Acceptance Tests (UAT)?
The acceptance criteria in the example above are based on the end user’s perspective, and the test is based on that user manually clicking through a user interface to complete the actions (or, if the test was automated, the test automation software would be simulating the user’s actions). Therefore, this might be called a User Acceptance Test.
However, not all acceptance tests are user acceptance tests, and not all acceptance criteria are based on the user’s point of view.
For example, during development a software development team may agree on acceptance criteria around how numbers are rounded for a given component of the application. A software developer may then write a unit test to ensure a given piece of code is rounding numbers based on the agreed industry standard. This is at a different, far more granular level than a user acceptance test, but it is still an acceptance test. If numbers are not rounded appropriately for that component and demonstrated via a unit test, the team does not accept that work as complete. This is an example of a unit test being used as an acceptance test.
In the Ideal Test Pyramid, User Acceptance Tests sit at the top and represent tests of the GUI and related processes. But acceptance criteria, and tests against those acceptance criteria, can be written at all levels of the pyramid. When testing and deploying an upgrade to enterprise software such as a warehouse management system (WMS), acceptance criteria are more typically seen at the Integration, System, and User Acceptance (GUI) levels of the pyramid, often documented in Cycle Features.
How are acceptance criteria and acceptance tests used and formatted?
Here is where things get truly interesting – acceptance criteria and acceptance tests are used in a variety of ways depending on the context. There is no “one right answer,” which can be liberating or frustrating depending on your viewpoint. Some examples of uses and formats:
- In contracts as a measure used for sign-off that work is complete
- In agile software development as a means of defining requirements of a work item
- As a business process used as part of behavior-driven development and/or business process validation
- As part of the creation of a formal Business Requirements Document (BRD)
- As an intervention in the middle of a project that is going sideways as a means to get it back on track
- As a way to define acceptable services and ways to measure those services (similar to an SLA)
- With or without User Stories
- In the format of Given/When Then
- In table format, for example with the columns “Action” and “Expected Result”
- In a checklist format
- From a technical perspective or a business perspective
- And the list goes on…
The one thing that all of the contexts and formats above have in common is: Acceptance criteria and acceptance tests are all designed to get agreement across collaborators and validate that agreement was met. That validation can be automated or not automated, depending on context.
In Summary, “Acceptance” involves collaboration.
Whether the people drafting acceptance criteria and/or acceptance tests are a client and a solution provider or a development team and a product owner, the goal is to agree to a common set of requirements that are written down and testable, and to then test them to determine the requirements were met. In other words, “acceptance” involves collaboration. Unless you are drafting a contract exclusively with yourself to change your own behavior – atypical in the business world but certainly not unheard of! – you are using acceptance criteria and acceptance tests to document and validate shared assumptions and meaning.
So, in summary: Don’t write acceptance criteria or acceptance tests in a vacuum!
Write them together, or at least review and refine them together. Revisit them often. Use them throughout development. Use them for reporting. Clarify them as needed, collaboratively. This is their intended purpose, and what makes them a powerful tool!
Are you interested in learning more about implementing test automation in your warehouse management system? Check out our success stories, more of our blogs, or learn more about the Cycle platform.
