Manage Learn to apply best practices and optimize your operations.

Get the most out of continuous testing with Jenkins

With dev cycles shrinking all the time, Jenkins’ capabilities continue to grow. With plug-ins at its user's disposal, Jenkins is effective at CI/CD.

Development and test teams still perform their continuous testing with Jenkins. Its open source nature as a build...

platform underscores the importance of companies and their developers productively using Jenkins for continuous code testing and integration.

Continuous testing with Jenkins enables a large and growing number of open source plug-ins that expand the product with new ideas. These extensions make Jenkins compatible with many previously unsupported languages and tools. Jenkins enables software to execute continuous integration/continuous delivery (CI/CD).

Implement CI/CD with Jenkins

CI/CD are offshoots of the Agile development methodology. CI continuously gives feedback, whereas the execution of CD is more or less scheduled for the end of each Agile iteration. Any change that is released into production must be somewhat coordinated.

CI is the process of compiling one's larger project, which entails individual programmers checking their code into a code repository, such as GitHub or Subversion. Accordingly, CD is the practice that pushes that code out to its target environments. CD includes infrastructure steps, like creating virtual machines and containers, as well as configuring networks and storage.

At each step of CI/CD, development and test teams can run tests. The assessments generally fall into the following categories:

  • Functional: Functional tests check that the whole program does what it is supposed to do from a functional point of view.
  • Unit: Unit tests check that individual subroutines in a program work.
  • Smoke: Smoke tests check the most important functions before pushing onto the further tests.
  • Integration: Integration tests check that various systems connect with each other.
  • Load: Load tests verify software performance under the requirements demands.

When the code passes the tests, Jenkins goes to the next step. When it doesn't, Jenkins can be made to stop. However, these assessments don't always provide Jenkins with the most comprehensive feedback loop. So, the software tool might know that a test ran, but it might not really know whether it ran successfully.

Select quality plug-ins to improve CI/CD. Specifically, choose plug-ins that push the code toward the target environment and enable event-driven feedback into the push process.

When Jenkins' dashboard lights up red, that indicates failure, and developers and testers can look at the console's output or Slack, if your team utilizes it. Slack is a messaging tool that lets teams chat with each other and pull in other resources, like code deployment and analytics data. It is often used as a place to report Jenkins build output. Slack keeps output in neat channels to avoid overwhelming the conversation with too much clutter.

Still, the output from these builds is often too verbose and large to understand at a glance. Tools like Sauce Labs, Postman and BlazeMeter make CI/CD tests easier to understand.

Continuous testing with Jenkins within Java

For programs written in Java, Jenkins typically runs through these steps:

  1. Developers create a phase in Jenkins that runs Apache Maven or Gradle to compile their code. These packages define dependencies, meaning external JAR files, that a program needs to include.
  2. Programmers then check their code into the code repository.
  3. A check runs that will let the programmers know whether the subroutines they have written will clash with what other developers wrote.
  4. Programmers also execute test code, written with frameworks like JUnit, to test individual subroutines. Meanwhile, functional tests check the whole program. This second approach is often called business-driven development and is made easier with natural language frameworks, like Cucumber.

To illustrate, imagine a unit test as a math equation. Suppose we have a function square (x) that calculates the square of x:

square (x) = x * x

That is in one program. The programmer also writes a second program that, in this case, says:

assert (square(2)) = 4

The second program tests whether or not the subroutines correctly calculate the square of two. If the result is not four, it throws an assertion error, and Jenkins will stop. In practice, natural language testing works more like:

if bank balance < $100 and withdrawal is > $100

This process is typically how a framework like Cucumber works. This particular testing automation tool is used in conjunction with business-driven development, an approach that has you write the test code first. With no logic written yet, Cucumber will fail on the first try. At this point, you would need to write the minimum amount of code until the test passes. So, the focus is to get the bare-bones pieces of the program to work upfront. This avoids wasting time on tasks that are not on the critical path.

Continuous testing can also test more than business logic or subroutines. The webpage and mobile app can also be tested using tools like Selenium. Testing tools emulate various browsers and browser versions with different mobile devices. Such programs can test, for example, when users click on the next button, the next screen comes up and not something unexpected.

The final step of continuous testing with Jenkins is to assess that CD works. This is often implemented using SaltStack, Ansible or other build and orchestration tools.

This was last published in March 2018

Dig Deeper on Agile Software Development (Agile, Scrum, Extreme)

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

When you were looking to expand your possible development practices, how did you decide between adding more plug-ins from a program in use or adopting a new application?