alex_aldo - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

6 ways to catch defects in software, tighten feedback loops

Face it: Bugs are inevitable, but quality improves when you find them sooner rather than later. Follow these procedures to circumvent lengthy and expensive feedback loops.

The later testers find defects in software, the more expensive they are to correct. This economic reality applies whether you're on an Agile team with rapid iterations or one that follows the Waterfall model of progressive phases. So, you want to find bugs as soon as possible.

The feedback loop -- particularly the test feedback loop between testers and programmers -- helps teams find, report and fix those bugs. The more a team tightens the feedback loop, which also includes end-user feedback, the better it can address defects in software and boost productivity.

Defects found late in software projects often undertake a roundabout journey to a resolution. Firstly, QA must estimate, track and, eventually, assign the bug to a programmer. However, that programmer must first finish other work before he starts to fix the bug, and he might not even be familiar with that particular aspect of the application's code base, causing further delay.

Compare that workflow to one where a QA tester makes a programmer, who inadvertently created a bug, aware of that defect the same day it was added to the source code. The programmer can address the bug quickly with the context for it still fresh in his mind. That's why a tight feedback loop makes all the difference.

Techniques to tighten the feedback loop

1. Reduce the distance between tester and programmer.

One of Agile's most transformative changes was to take testers from a separate group with a separate manager and embed them into specific development teams. Simultaneously, testing also shifted to focus on one feature at a time, instead of end-of-line system testing.

Think of testers on an assembly line of sorts; they take the immediate work from the programmer and inspect it. If the tester has a question, she can turn to the programmer and ask him directly. The goal is to discover the bug before the programmer moves on to the next feature. This arrangement is much better than to have testers in a different room, looking at all of yesterday's work and unable to notify programmers easily about defects in software.

Some teams I've worked with have testers ride along the programming process, meaning they flip between code and test frequently, and even contribute to code.

2. Reduce the work in progress.

Envision an overworked team: Programmers work on 10 things at a time, support multiple projects, hit roadblocks when they can't get answers to their questions and hit more snags when another pull request gets blocked. Nothing in that way of working is conducive to fast software releases.

Now, picture a team where programmers work on one thing at a time. When the tester evaluates a programmer's work, the latter waits for the former to fully trawl for defects in the software. This second team will have little to no switching costs and delays; in my experience, each individual task can get done as much as 30 times faster than in the hands of the overloaded team. While this is an extreme comparison, measure average work in progress, and cut it by 10% to 25%. Then, cut it one more time. The feedback loop will tighten, and you can fix software more quickly.

3. Reduce testing preparation time.

Some groups spend a lot of time preparing tests. They create test strategy documents, master test plans, test cases and loads of other documentation. Finally, when the test documents pass review, actual testing begins. All that work delays the development process -- it slows down the feedback loop.

A capable tester should look at the work done and create a test plan concurrent with the actual testing. Some argue that many testers are simply not capable of this sort of work -- that a QA lead must tell them what to do. But, if testers have the aptitude to multitask, train them; if they don't, maybe they don't have the aptitude to do effective testing at all.

Know your loops

A software development team can have one to three feedback loops at any given time:

  • a developer testing her own work;
  • a second person testing a feature; and
  • the final shakedown test before a release to production.

Additionally, some teams might have periodic automated tests, a CI run, performance testing and other types of test results that cause delays. When debating which loop to tighten, evaluate the value of reducing time-to-test results and the ease of tightening a particular loop.

To get a grasp of that, take a few pieces of paper, and estimate these lengths. Write down the process, and if you'd like, study them for a sprint. When finished, figure out how to slice off a piece. For example, you can skip tests that seem to never yield useful results. In some cases, these tests are redundant anyway. Investing in steps such as automating the setup process could have other benefits, like automating the build process as well.

Sketch it out, give it a try, look at the results and repeat.

4. Reduce environment setup.

If it takes hours to set up a test environment, testers will likely do that work once per sprint, which creates distance between development and testing. If it takes days, you might inadvertently be stuck in what is effectively a Waterfall model. Setup also slows down test loops and delays feedback.

So, figure out what takes setup so long. Once you identify the problem, reduce the delay with something as simple as a batch program to load data into a database or a push-button to create a server for any given build or branch.

5. Improve first-time quality.

Many defects in software are accidental. These mistakes happen because of a lack attention to detail or a misinterpretation of vague directions. When such bugs become too common, programmers can't advance to create new features. There will be more trips through the feedback loop in order to finish the project, and everything will take longer.

To fix this bottleneck, reduce the defect injection rate. Analyze the root cause of these missteps, and remove them to improve first-time quality. This shift often draws testers into the creation of software requirements.

6. Use appropriate test tooling.

Obviously, test automation helps reduce the test feedback loop, but teams easily overemphasize it. Human testers poke, prod and examine the software in a way that automated tests can't. Naive attempts to add automated tests on top of a legacy system will probably add cost and time -- and not much benefit.

Unit tests and test-driven development, along with a modest amount of automation, can improve first-time quality. For example, a light build verification test can help; just don't overdo it.

Dig Deeper on Software testing tools and techniques

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

How do you improve collaboration between programmers and testers?