Is there a need to change traditional testing approaches? What is DevOps? Who’s using it? These are a couple of...
the questions addressed by Manoj Narayanan of Cognizant Technology Solutions at STAREAST 2012 in his session, “Testing in the DevOps World of Continuous Delivery.” He explained the benefits of DevOps and how it impacts the organization.
Traditional testing approaches are geared toward sequential delivery, which leads to an emphasis on discrete skill sets and therefore minimizes cross-functional behavior. The sharing of ideas and mutually beneficial collaboration is limited as there are barriers between development, testing and deployment. Narayanan identified the drawbacks to the sequential delivery approach as a longer wait time and higher risk; essentially a developer may be saying, “I don’t know if what I’m building is what the end user wants.”
In order to meet the objectives of business – to reduce risk, respond fast and beat competitors – Narayanan supports moving from a discrete delivery approach to a continuous delivery approach. The language changes from “build it right” to “build the right it.”
Agile development and DevOps
Agile development offers one path to continuous delivery as it brings developers and testers together and focuses on iterative development. Yet, according to Narayanan, “the ‘last mile problem’ still exists.” The wall separating deployment from the development and testing side encourages a long time between testing and deployment, and furthermore, testers do not necessarily have a systems administration skill set.
DevOps offers a potential solution to this integration challenge, bringing together development, testing and operations all on the same team, a team capable of playing all the different roles.
“Like Agile, this approach aims at getting features deployment ready at a high frequency. But where Agile primarily focuses on the functional and non-functional readiness of the application, DevOps takes it one step further and ensures the Operational and Business readiness as well,” Narayanan writes in his blog.
Narayanan noted several websites that are currently employing the DevOps model, sites like Facebook, Etsy, Orbitz, Groupon and Flickr. He explained that the frontrunners have tended to be Web 2.0 firms with strong reliance on eCommerce where fast changes and improved response time are paramount. He mentioned that Netflix is involved in this space as well, actually going with a NoOps approach.
When it comes to testing, the change to the DevOps environment must be gradual, and the talent involved will need to acquire new skills and adapt to new responsibilities. Testers need to gain knowledge in development languages— which fortunately is a bit easier with user-friendly tools like Python and Cucumber – and they also need to learn deployment processes and tools. Developers and systems administrators, on the other hand, must learn about test processes, design techniques and related tools.
Making the change to DevOps
Shifting to DevOps also makes continuous integration mandatory, according to Narayanan. He explained that in test-driven development we expect to fail first, but those errors inform the next cycle of development and testing. He emphasized that teams must have strong discipline around the single source code repository. The build process can be automated, resulting in a fast build. In order for all of these elements to work effectively, he highlighted the need for transparency; everyone on the team needs to know what is happening.
The process is also affected by the heavy reliance on innovative automation, which is embedded early in the lifecycle. “’Smart testing’ is dissolving the boundaries of traditional system and integration testing,” said Narayanan. Teams can now “leverage an optimal mix of automation across the lifecycle,” which includes automated unit testing, automated service layer testing and automated regression testing. Continuous integration is further facilitated by release management automation.
Governance, technology and stakeholder buy-in
When it comes to governance, Narayanan holds that teams work best in pods which are comprised of ‘jacks of all arts;’ these pods are centrally coordinated and are characterized by ongoing communication and collaboration. The incremental changes taking places are driven by business goals. One result of this pod arrangement is that a greater need for specialists arises; they take on the challenging sub-tasks that the regular pod members may not be equipped to handle. Pod members can also identify repeatable tasks and automate them. This approach works especially well in smaller and mid-sized organizations.
The cultural impact of DevOps also appears in the technology the organization uses. DevOps teams need user-friendly tools that all team members—developers, testers and systems administrators— can learn and use. In addition, the focus on automation and reusability intensifies.
In order for stakeholders to buy in to these changes, communication is key, and stakeholder buy-in, likewise, is key to the success of this model. The success may be viewed in terms of speed and ROI—as the deployment happens faster, ROI improves.
For comprehensive conference coverage, see our Software Testing Analysis and Review conference page.
Dig Deeper on Test-Driven and Model-Driven Development