The whole point of DevOps is to get your product and subsequent iterations to customers quickly, while still maintaining a high level of quality and stability. But this requires the magic of automation, especially when it comes to testing.
Given that testing must be conducted every time developers change their code, there’ll always be a hefty testing load. All of that testing is obviously necessary if enterprises hope to consistently deliver a stable product, but it stands to be remarkably time-consuming. This is where programs like CA Technologies’ Service Virtualization and its Test Data Manager applications along with IBM’s Rational Test Virtualization and its Test Workbench tools have emerged as specific DevOps testing platforms.
Now, there will always be some need for manual testing since machines can't test for everything; things like usability and user acceptance tests can only be evaluated by human minds (for now!). But much of the testing that takes place in the deployment pipeline can be automated, including unit tests (after the build, which is automated too!), acceptance tests, and capacity tests. With programs like HPE’s ALM Octane or TechExcel’s DevSuite--two integrated lifecycle management systems--a DevOps team can schedule and integrate QA tests as the development process goes along, catching and fixing issues in a timely and inexpensive manner. This allows for speedy progress through the pipeline – think minutes and hours– and maintaining that flow is an important part of successfully achieving DevOps.
It's not just progress through the pipeline that is made more efficient; potential problems are identified, and feedback provided quicker than before. This allows the development team to address the issue and send the code back through the pipeline again sooner. Since the testing is taking place potentially in minutes (in the case of the automated unit tests) or hours (acceptance, capacity, and any manual tests that may take place thereafter), developers can count on rapid feedback to facilitate a faster overall turnaround time.
The benefit of automated testing extends into the future, as well. Automated tests support continuous learning, which saves the time and effort of having to conduct the tests, and the time to figure out how to improve processes. When a release candidate fails a test, it offers a learning opportunity. Wherever it fails in the pipeline, it gives team members a chance to think about how to improve what came before that phase (e.g. if the code fails the automated acceptance testing, it raises the question of how it got past unit testing and how those unit tests can be improved).