Risk-Based Strategies for Automated Testing
The warehouse environment is a delicate balance. Like the metaphorical butterfly fluttering its wings, the smallest detail can have the most profound (and often unintended) outcome.
That’s just one of the reasons why frequent testing of the systems any business uses is a must. Everyone from the C-suite to floor managers needs to visualize and understand the risks associated with changes to infrastructure, software applications, workflows, staffing, and more.
But many organizations want to test everything, all the time, with the philosophy that there is no room for error of any kind. Few, if any, companies have the bandwidth for this degree of certainty. And while automated testing solutions make it seem possible to “just test everything,” the reality is that such extreme due diligence would require not just automation, but also unlimited budgets, time, and human resources.
Testing powered by automation is a game changer for streamlining deployment, but it still requires the kind of high-level thinking only people can do.
An ostensibly “simple” change in the length of an employee’s shift may lead to a broad ripple effect.
So it’s important to know exactly what should be tested — what needs testing — before moving forward with changes, whether they’re for a weekend crew or an entire workforce.
So what questions should an organization ask — and what kind of testing makes an impact?
Crunch Time: A Risk-Based Test Matrix in Action
Even though a warehouse may have thousands of things to test, chances are only a fraction of them represent business-critical insights. Risk-based testing focuses the effort where it matters — on delivering the most critical information at the right time.
Here’s a typical scenario for a (hypothetical) Cycle Labs’ customer:
Demand is sky-high and a warehouse is inundated with new orders. Management decides to extend employees’ weekend shifts from seven to 10 hours. Supervisors tell pickers to log on to the ESS portal to add to their hours. It’s an everyday, mundane transaction — no big deal; nothing can go wrong… right?
But from a risk-based testing perspective, that couldn’t be farther from the truth. A straight seven-hour to 10-hour extension seems like a no-brainer when it comes to the data to input for testing. Yet the warehouse team is incredibly diverse: different ages, skills, length of employment, hours worked so far this month, and varying pay grades.
At Company X, even just considering three individual employee circumstances presents a variety of outcomes:
- Employee A has been working for the company for 20 years. A company veteran, he is significantly older than most of his colleagues (as is his union contract). When he inputs the change in his hours from seven to 10, the output result should be an eight-hour shift, plus two hours of overtime and an additional 15-minute break.
- Employee A’s locker is next to another member of the team, a contract worker who’s new to the company. Employee B gets no overtime, breaks, or penalty rates. He will simply work the extra three hours on his shift.
- The third employee, Employee C, is on a fixed salary. She usually works eight hours each shift and will be paid for two hours of overtime.
Without taking these factors into account, testing could produce multiple results that are all correct. Which begs the question, How does one pass or fail the criteria in a case like this? For an organization to test the outcome of the three-hour shift extension across its userbase, it must effectively look at more than one test.
Settle the Score, Know the Risks
When a business begins a testing program, factors like these will be part of the requirements-gathering phase when working with a solution architect who understands risk-based testing. Among the questions to consider:
- How complex is the requirement?
- Is this something we need to test?
- What is its impact of failure on the user?
- How often is this likely to fail?
- Is this part of code that’s really complicated?
- Is the requirement business-critical?
A scoring mechanism (whether it’s high or low, yes or no, one to 10, one to five, or any other schema) builds a matrix that illustrates the relative complexity of outcomes.
With just four outcomes, this is a simple case. More complex systems with deep embedded juristics or rules-based systems have dozens of different outcomes. Without a way to score factors (like overtime, for example), every release, upgrade, or hotfix means regression testing.
The scoring mechanism in the scenario above makes it easy to see high-risk factors and prioritize business-critical issues. By embedding this technique into the process of capturing user requirements, attaching them to a traceability matrix, and using risk-based testing, organizations can prioritize effectively in a way that’s still accessible to their project teams.
Flexibility is a critical component of any testing platform. Any must-have, can’t-fail processes should be tested regardless of their scores.
This strategy provides a clear framework for deciding exactly what to test and a way to communicate that information to business leaders.
Testing is a critical part of transformation. For testing to be truly effective, it should begin with developing a strategy in the early stages of preparation for any project. It’s not something that can be done in the midst of sprints — when outcomes are already being generated. Make testing strategy front and center, and the tests themselves will provide what matters most: confidence in putting an organization’s processes, platforms, and people out into the world.
Are you interested in learning more about implementing test automation in your warehouse system implementation? Check out our success stories, more of our blogs, or learn about the Cycle platform.
Test automation is the backbone of any successful enterprise testing strategy – especially as enterprise systems continue to rapidly evolve. However, implementing test automation requires commitment and investment. How do you justify it?
This calculator estimates how much money you spend on manual testing currently as a starting point for building a business case for switching to automated regression testing using the Cycle® platform.
