Ode to Jenkins
With February upon us, it is time to celebrate the 8th birthday of one of the most influential pieces of open source software, Jenkins. Few tools have become as ubiquitous across projects as this automation server. Jenkins augments software groups the world over by allowing easily built pipelines for continuously verifying changes to a project.
Continuous Testing
Creating tests only helps in the development process if they are run. It sounds trivial, but many teams fail to consistently run tests against their work if they run tests at all. Time constraints, limited resources, or even forgetfulness can lead to changes to a product being made with little regard for testing.
Automating these tests can help bring speed and confidence to any change to a piece of software. Computers are masters of doing repetitive tasks, and testing should be one of those tasks. By properly automating the testing of an application, a team receives faster notification of any breaking changes, as well as a lower cognitive load when working. Needing to remember how to execute the suite of tests, context switch to execute the tests, spending the time executing the tests, and reporting back-breaking changes can all lower the productivity of the team. By building automation into the testing process, kicking off a single job for the entire test suite is typically all that is needed.
Continuous testing takes automated testing to the next level. Rather than relying on any one person to run tests, tools like Jenkins run them automatically on a defined event. These events might range from whenever any change is committed to the repository, to running tests every night. At a glance, any team member can quickly see the result of any execution and how long the execution took with Jenkins reporting. Continuously executing tests along with the easily interpretable results aids the team in quickly pinpointing when a problem arises and narrowing the scope of what changes created the defect.
Since Jenkins runs on or across servers, all the work happens in the background without taking time or resources away from the team. This may not seem like a big deal when simply executing unit tests that run quickly and without the need for interactions with a graphical user interface; however, UI tests are exactly the opposite. They often take minutes, even hours, and require that no one is using the machine executing the test while the test is running. Often literature around continuous testing focuses on testing levels like unit tests, but for many teams, testing the software at the application’s UI level is the only viable option.
Cycle + Jenkins
The types of teams that rely on UI tests can potentially benefit even more from continuous testing than teams whose speedy low-level tests take mere minutes. If a team’s UI test takes hours to run, kicking that test job off as soon as possible is vital. Tools like Jenkins can assure maximum uptime on testing by kicking off the job, or multiple jobs in parallel across servers, as soon as any change is made.
This need to continuously test at the application level is why Cycle has been engineered with continuous testing in mind. Running a feature file, or an entire playlist of feature files, is as effortless as a single command to Cycle-CLI. Configuring these jobs is made simple, as Jenkins automatically knows the success of the execution by checking the exit code when Cycle-CLI completes.
The introduction of projects in Cycle 2.2 also provides easy setup and sharing of the settings necessary for consistent execution across machines. Running multiple branches or projects on the same box requires no extra configuration as the Cycle project file now lives inside the folder alongside any tests.
Optimization
Even though continuous testing can run jobs on a dedicated server for testing, some thought will still be needed towards resources and time. Large application UI tests could require many hours to complete. It might be necessary to run only a subset of tests upon any change made. Relegating a full UI test execution to running at night while only running faster more concentrated tests during the day can still provide valuable feedback on the stability of the project. If running a subset of tests is not an option or the nightly tests still takes too long, Jenkins can be configured to run tests in parallel across multiple machines. This can even be configured to only happen at certain stages of the job if others require no parallelization.
Conclusion
Like any powerful tool, Jenkins requires some initial learning and configuration in order to work optimally testing an application. Once the initial groundwork has been laid, the benefits of continuous testing far out weight the initial investment. Quick notification of any breaking changes to a project lower the cost of development as problems are found sooner. Coupled with Cycle, Jenkins makes running robust UI tests even easier. It is no wonder this little automation server has helped redefine the software development process.
Are you interested in learning more about implementing test automation in your warehouse management system? Check out our success stories, more of our blogs, or learn more about the Cycle platform.
This post was written by:
Cameren Dolecheck
Software Developer
