What Makes a Good Test Case?
We get distracted and are often hurried when designing test cases, as everything is on a tightly project-managed deadline these days…but are they good? What makes a “good” test case? Let’s start by level setting on a definition of a test case. We define a test case as a set of conditions, variables, and/or actions that are performed on a system under test in order to validate that it meets requirements and verify that it functions correctly.
Good test cases have…
A clear objective with refined scope
What specifically is the intent and scope of the test? Is this a white or black box test, and is the purpose regression or performance? When determining the test objective, start at a high-level considering user context and then work down to thinking at a granular functional level. For example, are we only testing that a single login button was clicked or are we testing that an existing user can log in to the website? There is certainly overlap between the two, but both are not necessarily the same test. Don’t hesitate to bring in more of your team at this stage to get their input; consider getting everyone together for an exercise like example mapping. If you botch the objective, which is really the overall point of the test, then all the work related to that test case that comes afterward is a waste of time.
Obvious and meaningful pass/fail verifications
What constitutes a “pass” and “failure,” and how are both determined? Each should be clearly defined as specifically as possible. Using the login example mentioned above, are we satisfied that an HTML “screen read” verification that sees “User Logged In” is designated a “pass”? Or, should our test case check a user log file on the web login server? Here is a more high-level (though admittedly morbid) example: Let’s say we want to test whether the guy in the graphic below plunged to his doom or successfully makes the jump to the other side. We can check the target ledge to see if he is there, but what if he isn’t? Is the guy not being there a verification that he plunged to his doom? Something we didn’t prepare for could have happened, like a giant pterodactyl might have snatched him up mid-leap or maybe our hero spontaneously combusted. The only way to know that he plunged to his doom is to check the canyon floor for his crumpled body. The test case is only as accurate as what conditions you are verifying or validating.
Clear and concise documentation
Create a standardized template for your test cases recording things like a unique ID number, description, any pre-conditions, related datasets, and expected results. This is especially important for manual testing and/or if your test scripts are written in low-level code or are otherwise hard to read. If you have a test automation solution based on a business readable scripting language, then the readable scripts could potentially double as a test case template and related process documents.
Traceability to requirements
All test cases for a system under test should be traceable to the business requirements. We want to ensure that we have full test coverage of requirements and change requests while also ensuring that we are not wasting our time testing irrelevant components. Consider using something like a requirements traceability matrix to help with this effort.
Reusability
Test cases will probably change over time as the system under test evolves throughout its life, but we certainly want to write automated tests that will be reusable for as long as possible. Write the test case to be modular and easily maintainable. Let’s say the login example test case will be automated, if I know the button image will be changing in future versions then I should consider interacting with the web element instead of the image in order to make the test more maintainable and in-turn, more likely to be reused.
Independence from other test cases while testing one thing
A single test case shouldn’t depend on other test cases for execution. Ideally, to create larger end-to-end tests you should be able to combine your independent, modular test cases into test suites for sequential or parallel execution.
Permutations taken into account by the test case designer
Is it necessary to test every permutation or just the critical path? What about negative testing possibilities? In our web login example, we want to test valid username and password login combinations – but do we also want to check for the possibility of various kinds of invalid login credentials? What if users enter weird characters into the username and password fields? It might be ok to only test the critical path, especially if a tight deadline is looming, but you should at least have the debate where to draw the line with your testing.
Are you interested in learning more about implementing test automation in your warehouse management system? Check out our success stories, our blog, or learn more about the Cycle platform.
Test automation is the backbone of any successful enterprise testing strategy – especially as enterprise systems continue to rapidly evolve. However, implementing test automation requires commitment and investment. How do you justify it?
This calculator estimates how much money you spend on manual testing currently as a starting point for building a business case for switching to automated regression testing using the Cycle® platform.
This post was written by:
James Prior
Technical Pre-Sales Consultant
Evan Edwards
Former Chief Technical Officer
