360 Guide:

Advice and tips about DevOps for beginners

carloscastilla - Fotolia

Get started Bring yourself up to speed with our introductory content.

A comprehensive beginner's guide to DevOps for testers

Newcomers to DevOps have a steep learning curve and nowhere more so than in test. Expert Matthew Heusser has a beginner's guide to DevOps and testing.

People brand-new to DevOps often come in with a negative bias, particularly when it comes to software testing. Automation is a large part of DevOps for testers, but it does not eliminate the need for humans. What a move to DevOps does mean, though, is that testers must prepare to rethink this classic role. Testers must become risk managers. The testing work will not go away, but who does which job and when will shift.

How DevOps will change roles

As early as 1999, the advent of extreme programming suggested the end of the testing role in favor of continuous pair programming. At that time, a tester was more of a checklist-follower, and the thought was that automated tests written by a pair could replace that role. Since then, other ideas have evolved. Those include a single tester per developer pair, one test "coach" for a much wider group or a test team that assumes some of the build, merge and CI activities. Another option is to have very few testers but allow a programmer to play the "test" role if there is no tester available. Finally, some software organizations that support a wide variety of platforms (think Android, iOS and web) have turned to crowdsourcing and advanced beta programs to find bugs. In those cases, a few testers organize and oversee the test activities.

In her book, A Practical Guide to Testing in DevOps, Katrina Clokie wrote: "An organization might consider removing a tester if they believe that testing activities can be conducted by other roles within the development team or by the users themselves in the production environment. Though there are no definite rules for determining whether a tester can be removed, I see five areas to consider: context beyond the team, support for quality, team dynamic, measurement and bias."

Where the programmers take an idea from concept to cash, testers can follow along, if they can add value at every step along the way.

The common thread in all of these different ideas about software testing is a move toward one-piece flow. Once a story begins, it continues until the work is done, with only breaks for lunch, team meetings and going home at night. To do that, the technical working group needs all skills on deck -- from test to code to the database administrator to a little bit of operations.

When it comes to DevOps for testers, it is all about the value add. Testers can add build responsibilities or be involved in requirements, monitoring and rollback. Where the programmers take an idea from concept to cash, testers can follow along, if they can add value at every step along the way. If testers can contribute to one-piece flow, their role might be needed. As Clokie implied, if not, they might not.

Before you begin

To approach the DevOps for testers process for the first time, it's vital to analyze the current risk, how to manage risk, how threats might change with more frequent delivery and what the ideal end state will look like.

Of course, there is no ideal end state. Not only will the ideal end state change before the team reaches it, but once there, another higher mountain will loom. But an ideal end state gives the team a direction for improvement. If the team wants to ship more often, a regression test or a preflight check might be eliminated or reduced. If you eliminate checks, in turn, it might suggest a change in architecture to enable microreleases rather than working with a massive code base.

devops and testers

Another consideration with DevOps for testers is to determine who will be testing. If the entire team is involved in test, what is the mechanism to hold that together? If programmers will write all of the automated tooling, what skills will they need? If testers will write tooling, how can they integrate with the rest of the technical staff to enable one-piece flow?

One way to tackle this problem is to make a list of techniques and assign responsibilities appropriately. A better option is to take a further step back: Make a list of possible bugs, including security and performance, and decide how those risks will be managed. Then, address who will perform each activity in the end state.

Finally, consider the tooling and infrastructure that need to be built. Many teams start with the automation, but it is smarter to build the tooling (things like push-button environments) first. The DevOps for testers role at this point is to act as the customer or product owner for the build pipeline and monitoring systems.

Where to begin

At this point in DevOps for testers, the changes may feel overwhelming. Testers can't just switch to DevOps, especially if the build/test/deploy tooling infrastructure is not in place. Instead, identify a single target condition, or stretch goal, and iterate toward that.

The target condition is generally a combination of the largest gap. It is the gap with the most opportunity to add value to the process, but it is also the easiest to do. In other words, start with something that will give the most bang for the buck. Teams rolling out DevOps for testers generally start with infrastructure and tooling, which is fine as long as waiting for builds, deploys or environments is the single largest issue stopping the delivery organization.

Automation and tooling

Many organizations target the GUI for automated tests early in the DevOps process. The team needs to create a large number of tests, like a comprehensive web, to catch every bug. But if the problem is too many bugs, that's not a test issue; it's an engineering problem. A better plan is to focus on tools to accelerate the build and deploy of environments. The build tools make it possible to go from commit to test in five or 10 minutes.

The opportunities for DevOps testers

Exactly what DevOps for testers means and how it will work will be up to the organization. The first quick win is often the ability to create an environment that contains only the changes for a very small piece of work that can be tested. That step represents the desired one-piece flow from dev to test. Testers may also help with requirements at the beginning and act as a sort of junior product owner during the story. These new roles may be sufficient to keep their jobs relevant. Others move into the ancillary roles mentioned above. In any event, a DevOps for testers effort is a chance to get all technical staff to stop thinking of test as a boring checklist activity best done by someone else and instead to think of it as a continuous set of activities designed to reduce risk. It must touch every role.

Dig Deeper on Agile, DevOps and software development methodologies

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

What was the most helpful step your organization took to begin the DevOps process?
Somewhere in all this has been lost the hard-learned lesson that professional testers are significantly better at detecting defects than others without such specialized skills and knowledge, and especially better than the developers (including analysts, designers, and managers as well as programmers) who actually create all the defects and thus are woefully unlikely to find their own errors. If your testers are not better, it means they’re not up to their role, probably because of deficiencies in your overall software processes, not that the others magically have gotten really good at testing.

I agree, Robin, though some organizations claim to have a great deal of success transitioning to having fewer testers who are more like coaches. I believe that model is worth discussing.
To me DevOps is simply the latest buzz word for what software engineering and its testing of has always aimed for, maximum process coverage with minimum effort. To answer the original question; my recent experiences in following Agile scrum combined with a change of operating system that provided repeatable quick deployment from test to production were the biggest steps to improved DevOps. Scrum development brings the need for inclusion of all system operatives and newer tooling brings speed of change. So even a few changes to an organisations SDLC can improve DevOps coverage. Operation teams can no longer claim they are not included.
I hear you man. On the other hand, one thing that is new is the combination of virtualization, containerization, the ability to 'script' environments, and cloud-based services. Put all those together and you get the ability to compose environments with code, perform builds with code, push code to production ... with code. We just couldn't do that 20 years ago.

I totally agree though that it just takes a few tweaks to turn a scrum organization that is NOT cooperating with operations into a much more effective organization. On the order of 30% increase in productivity by turning your head and squinting.

Okay, it's a little more work than that, but not that much! :-)