Writing Good Test Cases

Writing good test cases is harder than it should be. The most important skill for testing is arguably common sense – knowing where things will go wrong – but applying that common sense to repeatable procedures can yield spectacularly bad results.

I can’t help but dread opening the test repository whenever I join a new project. Perhaps I’m just jaded, but I expect to find one of two situations: either the repository is sparse with minimal content, or it is full of bloated test cases that make no sense. I expect to spend hours, if not days, trying to decrypt the intended behaviors from the procedures or from my team members. Anything better is a welcome surprise.

Why is this the case? When writing tests, I think most people emphasize the mechanics of testing rather than the exercise of product behavior. Ultimately, test cases should be living specifications of the product’s features. They should reveal how the product should (and shouldn’t) function. Each test case should cover exactly one main behavior so that the team can set clear expectations for its purpose and its outcome. Adding trivial steps or covering multiple behaviors together muddles the intention behind the test.

Whenever I write test cases, whether by myself or collaboratively with others, I follow a three-step process: Describe → Dissect → Define. This progression helps me channel my common sense into well-formulated test specifications. Let’s run through an example together.

Step 1: Describe

Suppose we are testing a Warehouse Management System (WMS) for an automotive parts retailer. The company purchases car parts from various suppliers, ships the parts to their warehouse, and stores them in inventory until they are sold online. Goods are received into the warehouse using an inbound order.

If we want to test the process of creating an inbound order through the WMS web client, we could describe the behavior in plain language. No code, no keywords – just a paragraph describing how things should work. Here’s an attempt:

Log into the WMS web client as a warehouse clerk. Navigate to the Inbound Shipments page, expand the Actions dropdown, and select the Add Inbound Order option. In the window that pops up, enter the following information about the order:

• Inventory number: INV1234
• Supplier number: VW1970
• Part number: 34PICT3
• Quantity: 5

Save the order. The window should disappear. Then, the table on the Inbound Shipments page should display the new inbound order with all the data that was just entered. To make sure the data is actually saved, check the database for the new order, too.

Anyone with common sense could read this description and understand the behavior in action, even if they don’t know anything about warehouse management systems. Writing rough descriptions may seem like “puking on a page,” but it’s a helpful way to kickstart ideas unhindered by rigid rules that could cause writer’s block.

Remember, software is built for people. If we can’t explain what it does with plain language, then nobody will be able to use it!

Step 2: Dissect

Let’s take a closer look at the description we wrote for creating an inbound order. The workflow naturally fits a step-by-step procedure. We could outline it with numbered steps to make it clearer:

1. Log into the WMS web client as a warehouse clerk.
2. Navigate to the Inbound Shipments page.
3. Expand the Actions dropdown.
4. Select the Add Inbound Order option.
5. In the window that pops up, enter the following information:

a. Inventory number: INV1234
b. Supplier number: VW1970
c. Part number: 34PICT3
d. Quantity: 5

6. Save the order.
7. Verify the table on the Inbound Shipments page displays the new inbound order.
8. Verify the database contains the new order.

Now, that’s less like a novel and more like a recipe! We removed some superfluous words, but the procedure is still the same.

The steps naturally fall into three distinct phases:

Arrange the system to be ready for testing:

1. Log into the WMS web client as a warehouse clerk.
2. Navigate to the Inbound Shipments page.

Act upon the target behavior:

3. Expand the Actions dropdown.
4. Select the Add Inbound Order option.
5. In the window that pops up, enter the following information:

a. Inventory number: INV1234
b. Supplier number: VW1970
c. Part number: 34PICT3
d. Quantity: 5

6. Save the order.

Assert the correctness of the outcomes:

7. Verify the table on the Inbound Shipments page displays the new inbound order.
8. Verify the WMS database contains the new order.

 

Arrange-Act-Assert is the pattern for writing a good test case that focuses individually on one behavior. A test arranges the system to be ready for testing, acts upon the target behavior, and asserts that the outcomes of the behavior are correct. Any functional test can be written following this pattern. It makes the test case clear, understandable, and executable. If the test fails, the reason for failure (meaning, the problematic behavior) is obvious.

When we dissect our test description, we should either discover the Arrange-Act-Assert pattern already present, or we should rewrite our description to fit the formula. If we discover that our description covers multiple behaviors, then we should split it into multiple test cases.

Step 3: Define

Once we have a clear understanding of the test case, we can define it with more formal language. I like to define my test cases using Gherkin, a special language that just adds a little bit of structure to the steps we’ve already written. Gherkin sets a standard for the whole team. Its steps are still written in plain language, and its syntax has a very small learning curve.

Here’s how our inbound order creation test could look with Gherkin:

It’s essentially the same verbiage, just with a few tweaks:

• The test has a concise title denoted by the “Scenario” line.
• Given-When-Then keywords denote Arrange-Act-Assert phases.
• Steps are written with subject-predicate phrases to denote who does what.
• The order information is written in a text table with pipe characters (“|”).

This scenario is very dependent upon aspects of the web UI. That’s fine if we want to explicitly test the web client, but changes to web pages would invalidate our scenario. Oftentimes, it’s better to write our tests in terms of business logic. Here’s how we could rewrite this scenario without explicit references to UI elements like the “page” or the “dropdown”:

We could go even further and eliminate all mention of the web client. It all comes down to how prescriptive the team wants to be in their specifications.

Gherkin provides standard syntax for writing tests. Its language frames the target behavior. The steps provide a clear example of how inbound order creation should work. Our test is no longer merely a description but a robust specification.

Another advantage of writing tests with Gherkin is test automation. Behavior-driven development (BDD) frameworks like Cucumber and Reqnroll enable testers to turn Gherkin scenarios into automated tests by providing step definitions in a programming language (like Java and C#). The Cycle Platform also uses a Gherkin-like language called CycleScript for specifying and automating tests. However, even if a team does not intend to automate their carefully-defined scenarios, using a language like Gherkin can nevertheless help keep test definitions understandable, unambiguous, and undistracted by separate behaviors.

The Magic of Formulation

Writing good test cases is an activity of Formulation, one of the three major phases of developing good test suites. The Describe → Dissect → Define pattern helps us refine our ideas about testing into robust, repeatable, readable specifications. If we are not sure about the behaviors at play, it can help to write (or even talk) them out.

Be careful not to get stuck on definition, though. Some folks hit writer’s block when they start using special keywords or structures. They want to do it “right” but don’t feel like their specs are good enough. Here are a few pointers for writing good definitions:

• Strictly follow the order of Arrange-Act-Assert. If you want to repeat phases, that’s a sign that you are covering an additional behavior that should have its own test.
• Keep the test scenario short. The full spec should fit on one page or in one view. If a scenario has more than about 10 lines, ask yourself if it is doing too many things.
• Avoid ambiguity. A scenario should answer questions, not raise them. 

Remember, intention is more important than perfection. Follow this formula, and eventually with practice, you’ll start writing great test cases!

This post was written by:
Andrew Knight
Principal Architect