Decoding Test Automation Buzzwords

Let’s deconstruct and decode hype in the realm of test automation, and determine what buzzwords like “codeless” and “scriptless” really mean. The goal of these buzzwords, in addition to making a sale at the end of the day, is to relay to potential customers that they don’t have to be programmers and can start writing tests quickly. We find the terms “codeless” and “scriptless” misleading because they haven’t done away with “code” or “scripts” – and more importantly – they indirectly present a false premise which is that the alternative MUST be code. There is another alternative, and that is business readable specifications (aka feature files) which are built collaboratively. We’ll also dissect the other usual suspect buzzwords/buzz phrases like “artificial intelligence” and “DevOps.”

Codeless or No-Code

The term “codeless” is “code” which means the tool’s syntax is high-level, which is nice, and buried in some kind of interaction editor, which is not nice. Interaction editors (and similar methods of selecting syntax) make things easier for the user initially, but over time it’s faster to just write it out rather than hunting and pecking through a long tree of syntax choices. Basically, it’s quicker to type the word “Click” than to scroll through a huge dropdown list and select it. More importantly, fully understanding the entire process in the system under test is valuable.

Scriptless

Test automation tools with recording features frequently boast that they are “scriptless,” but the claim is bogus. What is a listing of recorded actions? What would you call the written-out chronological sequence of actions in a movie? A script of course! Often after recording the various user actions in your test, there is a need to modify it, perhaps making a slight change to the process flow or maybe you want to work in some variables, and to make these changes you will ultimately work with a test script. So “scriptless” actually means “scripts are presented in a high-level, flow-charty manner.” A user has to click on each interaction (or step) in the flowchart to see what’s really going on. The recorded script is made overly concise and “pretty” at the expense of being able to see everything clearly at one glance and obscures the details of the process to be tested – all of which are of immense value to stakeholders.

My other issue with this claim is that you should EMBRACE readable scripts! As opposed to obfuscating and hiding everything in cells or flowcharts, business readable test specifications (or as it’s known in the BDD world, “features”) are fantastic for collaboration as they act as a single source of truth where everyone from the front-line user to the developer can work together. It also makes creating and maintaining tests much easier.

We’ve created a small web test in BDD feature file form using our solution called Cycle, and then made the same test using the recording functionality in a “scriptless” test automation tool. We have the recorded script on the left where we’d need to dig deeper to see the details of each interaction/step (most recording tools handle this via clickable icons which are not pictured), versus a Feature File on the right where every step of our test is both clearly and entirely laid out for your cross-functional testing team.

Artificial Intelligence

Whether or not “AI” is generally misused when marketing testing tools really depends on your definition. We’ve seen enough sci-fi flicks to have very high standards for the term. Once and awhile, we are quite impressed by the “AI-ish” self-healing and smart recording functionality that we’ve either seen demonstrated or tried ourselves, but we do not consider those features true artificial intelligence. We are always curious to drill down and find out more specifics when we see the most popular two-letter acronym in test automation used as part of a pitch.

Integrated Suite (of Testing Solutions)

This one is owned by larger companies in the testing space that often acquire technology rather than building solutions from the ground-up. On the surface it sounds like the user is in for a seamless experience with a variety of complementary tools, but in reality: The user ends up dealing with a dis-jointed, un-streamlined suite of overly piecemealed out programs with overlapping functionalities, different interfaces, and in some instances, the user is even forced to import/export between each tool to take advantage of functionality that really should be in one clean interface.

Train a Bot

We haven’t found this phrase to be technically false, or even hyperbolic. It’s just so drastically less cool than it sounds. While playing around with a popular robotic process automation (RPA) tool, we were told that we would be “training a robot” and for a half-second we imagined ourselves sparring with a BoxBot 5000 and teaching it our clumsy jabs, hooks, and the occasional sneaky overhand right.

In my head:                                                                                                                             

In reality: I hit “record,” clicked buttons and/or entered stuff, and then pressed “stop record.”

DevOps

No buzzword is more shoehorned into everything in the realm of test automation, and arguably the tech sector in general, than “DevOps.” We find the term is frequently used in a vague manner more often than being misused, as it’s hard to dispute that a tool which claims to “support DevOps” does not because DevOps encompasses such a huge scope including development, operations, and the merging of both. Any good test automation pitch should show its work when alleging that their tool supports DevOps.

In Conclusion

No matter what abstraction, buzzwords, or features attract you to a particular automation solution, true change can only take place if you are ready to challenge the cultural hurdles that will inevitably be faced. You need to ensure your entire team supports the change you seek to make. You need to recognize that automation takes strategy and there is value in choosing the high-value processes to be tested. No matter what tool you choose, bolting it on to an existing process and trying to automate everything isn’t going to result in success.

Despite the need for consumer advice regarding the abundance of too-good-to-be-true claims out there, the future is extremely bright overall in the test automation space and at Cycle Labs, we are on top of where the industry is headed – and what users genuinely want to see in their test automation solutions.

Have you seen a questionable claim somewhere in the test automation space? Contact us and let us know!

This post was written by:
James Prior
Technical Pre-Sales Consultant

James has been working in software pre-sales and implementation since 2000, and has more recently settled into focusing on technical pre-sales.