Regression testing is a type of software test that assesses if changes to an application, or other related software components, introduce defects. A quality assurance (QA) engineer performs these exercises to see if modifications to code break or hinder the way in which the application works or how it consumes resources.
An application change or addition can cause unintended side effects, called regressions, and might even pop up in components or systems separate from the altered code due to dependencies. A regression test flags these adverse effects. An application change or code modification that necessitates regression tests can include:
A QA engineer executes a regression test at any of four test levels:
Typically, a regression test occurs late in each test cycle, close to release; that said, don't discount the importance of unit regression testing, which narrowly addresses one unit of code at a time and reruns unit tests.
Regression testing must keep up with the speed at which an organization delivers software. Regression testing can be time-consuming and repetitive, requiring hours or days to complete. For this reason, a software team might choose to automate tests, both to reduce test execution time and free up the workers for other tasks.
Test case management helps a software development team determine whether a regression test is effective. A team assembles or orders its collection of test cases into a test suite.
One approach to regression testing is the retest all -- not to be confused with retesting -- technique, which, as the name suggests, entails the execution of every regression test case the team has written. While this method is thorough, it might be overkill for smaller releases. There are contexts that call for the retest-all technique, including when software is adapted for a new platform, language or culture, and when an operating system receives a major update.
If a software development team opts to build a tailored test suite for each test cycle, they can choose test cases according to their place in a priority system. A testing team might define a high-value test case as, for example, one with a high failure rate, or one that pertains to end-to-end or customer-facing aspects of the software. A test case that relates to added functionality, or checks critical application features, might also receive high priority.
A software development team can also follow a partial regression strategy, where they run just select test cases, but include tests related to any big and important features. This sort of approach is referred to as regression test selection (RTS) by some. For those who see a distinction between the two terms, with RTS a team only executes test cases that might be affected by -- or associated with -- the most recent changes to the code; with partial regression, the test suite also contains those tests which pertain to critical features.
To determine how widespread of an effect a change or build has on the software, the team can perform an impact analysis. Accordingly, an impact analysis informs the collection of test cases for the subsequent test cycle.
A smoke test, which typically precedes a regression test, only determines if the basic functionality of an application works. Regression test cases examine the software more exhaustively.
Whenever a test elicits a previously undiscovered defect, a tester should write a test case that similarly induces the bug.
An IT team can use a tool or platform to execute and optimize regression tests. A test management tool helps testers apply script automation, as well as trigger smoke and regression tests. In most cases, a regression testing tool leaves some level of manual test work for QA.
The regression testing tool market includes options with support for different programming languages and CI servers -- and by extension the CI/CD process. The market also ranges from low-cost or free open source options to feature-heavy commercial tools.
Open source regression testing options include Watir, Sahi and Selenium.
SmartBear's TestComplete and Ranorex Studio, two commercial options, test web, mobile and desktop applications. TestComplete is a front-end and functional testing platform, while Ranorex Studio is a GUI test automation framework. Each product integrates with a variety of CI systems. Functionize uses machine learning as part of its test automation capabilities and also integrates with CI/CD tools.
28 Jul 2019