Three Things To Consider When Choosing A Test Automation Tool
Testing has become mission critical for organizations trying to keep up with consumer demand. While the Amazon effect is long in the tooth and well-known, COVID, stay-at-home culture and even work from home trends have lit supply chains on fire. Sourcing, planning and the ability to physically adjust the supply chain, and in particular warehouses, must be flexible to meet consumer demands. The pace of change, the ability to adapt rapidly, is now either a competitive advantage or a major barrier to continued growth and competing in respective markets.
Massive investment has been flowing into automation for over a decade and looks to continue deep into the next decade. Hardware focus led the charge with more sophisticated conveyance and sortation followed by autonomous mobile robots, automated guided vehicles and now picking arms and vision sensors. As hardware typically leads, software follows. Sequencing algorithms, decision making intelligence (AI) and machine learning in general are now in play.
Why does this matter to test automation and the tool selection process? Because, warehouse management systems (WCS, WES, WMS) are under assault. Changes, large and small, are in need of being added on a regular basis. However, many of these systems are still being upgraded from legacy versions. Processes, tools and people are not yet aligned to continuously upgrade the systems in our industry nor are they flexible enough yet to absorb the required increase in the rate of change.
As an IT Director, I found bridging multiple perspectives a major key to success. How do I deploy my limited budget and resources to get the most bang for the buck. Blending company and operational needs with a more technical perspective is key to ensuring your investment is well-aligned to all aspects of the business. Making tool selection a business-decision and not just an IT departmental decision is the key to then leveraging the tool and solution you select.
Test automation is a pretty big industry in and of itself ($12+ billion and projected to more than double by 20241). According to Gartner, there are twenty that make their test automaton magic quadrant and even more in the mix when considering those platforms that specialize. Even our own Cycle automation testing platform leverages open source software as part of its core technology stack. This core testing software though, is just part of the total testing solution supply chain leaders require to add the needed agility to changing their systems in order to keep up with consumer demand and competition.
Without automating testing, how do the IT and operations leaders put change into their production systems rapidly without taking on unnecessary risk? There has to be a balance between speed of change to iterate against market demands and the risk associated with those changes. Shutting down shipping for even minutes these days can have both
What’s the difference?
To upgrade and make changes faster, you need to automate testing of the PROCESS, not the software. The process to be changed in operational environments includes the SCM software, yes, but it also encompasses the people and physical components as well. To thoroughly and effectively test the process, you need a tool that can test it fully: people through simulation, devices, data, software screens, databases, etc. The Cycle test automation platform does just that.
Of course, I am biased, I work for Cycle and my team deploys it daily. But...I also sat in “your” seat. I have been a customer, and I have had my feet over the fire. I’ve deployed warehouse management systems, I’ve designed and deployed WESs, upgraded hardware solutions, and implemented automation big and small. Through these experiences, I’ve boiled down all of the considerations into three key points to success:
Key #1: Consider what you’re testing, and why
Testing software for regression is important and your software team is likely already unit testing their changes or custom code. Hopefully, your software vendor is thoroughly testing THEIR software as well, before they push their 2-week release to you. This is the minimum expectation and par for the course when it comes to software or solution development. Consider what software teams, in-house or your vendor, are testing. They’re testing software as THEY expect it to work. They aren’t testing the software in YOUR environment, unless you’re lucky, nor are they testing across YOUR operational process. When selecting a test automation platform, make sure the toolset you choose (and the vendor) can test the touch points across your business process: people, devices (handhelds, other hardware), expected behaviors and the software they all integrate with. Automated testing delivers the best ROI when you have a plan and test business process. Ensure you’ve considered all of the test phases you’re trying to address with automation (functional, performance, etc.) as well as the scope of your test automation (end-to-end business processes, rainy day scenarios, etc.) to certify you’re ready for prime time. Imagine being able to test all of the above, automatically, whenever you want and far cheaper and faster than a manual approach.
Consider the testing pyramid and where your needs lie against current toolsets and the test automation solution you seek. You probably already have tools and team members focused on the base of the pyramid; but, testing processes, in an automated fashion, will take you into the middle to the top of the pyramid as displayed below. When working with our customers at Cycle, we typically target the top three layers for automation and see these types of testing delivering the most ROI.
Key #2: Consider the skill sets on your team
It doesn’t take an army to implement test automation, but it is important to consider skill sets. I find there are four key areas that make the difference when automating testing and having your team and vendor aligned to these will be the difference between success and expensive false-starts or failures. Environment readiness, documentation readiness, data readiness and people readiness are cornerstones to every successfully deployed test automation solution, and it starts with the team you put together. Picking a vendor solution that both compliments your team and adds value or takes your expertise up a notch is paramount. The number one reason a software solution fails is misalignment of people and expectations. Make sure your skill sets and the people you invest in are a top consideration when selecting a test automation tool and implementation vendor.
Key #3: Consider what you need to and CAN test
Why are you testing and what are you trying to protect? Don’t over spend on testing “everything”, at least not out of the gate. Start with mission critical processes and lean on your resident “experts”. Your people know your business and operation best, let them help prioritize what must be tested versus what are nice to haves in regards to test coverage. There will never be enough time to automate testing of everything and it isn't necessary with the right strategy. Consider what can be tested and the cost to do so. Find an automated testing solution that combines a service group that understands your world. A test automation strategy is strongest when testing expertise is combined with an understanding of your operations, and the technology running it. Not all test automation solutions or vendors are able to combine these key ingredients.
To Wrap it up
Test automation is simple at the core and similar to the automation that has been stealing headlines for the better part of the past decade: Do more, smarter, with less people required. But, like all automation, each individual tool or solution is not the same. What scope does the automation cover? What challenge does it solve? What pieces of the puzzle does it not solve? More tactically, but as important, leaders must consider the skill sets, timelines and costs required when choosing a solution.
Have you seen a questionable claim somewhere in the test automation space? Contact us and let us know!
This post was written by:
VP Product Delivery
Matt has a long tenure in the WMS industry. He has been a customer, an implementer, an IT Director, and now the VP of Product Delivery for Cycle Automation.
Global Engineering Group Upgrades WMS with Tryon Solutions & CycleOur customer is a global, high-technology engineering group founded in 1862. The company is a leader in tools and tooling systems for metal cutting; equipment, tools and services for the mining and...
Prominent Food and Beverage Customer Performs Regression and End-to-End Testing with CycleOur well-known food and beverage industry customer founded in 1897, has an extensive portfolio of consumer food and beverage brands. The company reported more than $7 billion in...
Essential Checklist for Deciding Which Tests to Automate To automate or not to automate?That is the question a lot of test professionals ask themselves- when deciding what tests should be run by software and which ones are best performed by people?To help, we've come...
What We Covered: High level explanation of Cycle as a behavior-driven test automation solution Overview of advantages including collaboration, lower maintenance, streamlining, and how Cycle is used to mitigate deployment risk Overview of Cycle interface, syntax, key...
The retail landscape is transforming--do you have the people and processes in place to excel? In this whitepaper, we outline the 5 ways to succeed in this consumer-driven landscape.