Four ways to reduce software testing cost without sacrificing quality

Matt Heusser, lead editor of "How to Reduce the Cost of Software Testing," shares a few of his favorite tips on how to reduce testing costs without sacrificing quality.

Say a company believes it is paying too much for testing. Now, if the bottleneck for general software development is testing, then cutting test staff will decrease your costs, but at the risk of introducing poor quality into your products. However, the sheer number of testers, compared to developers, seems unrealistically high compared to your own experience. What is going on here and how can you fix it?

Here is a four-step process to reduce test cost. I have used it. It works. 

Tip one: Manage by walking around and listening

First, let's walk around and actually look at the testers. Are they at the keyboard, actually working, running the software, installing it, making plans and updating them, or are they doing ... something else?  

The something else is the killer.

It might be supporting some other application, troubleshooting the current production version, waiting for a build, waiting for a server, waiting for a decision maker to decide what the "right" thing was for the software to do. This tends to manifest in lots of time in email, or hanging out at the coffee pot, water cooler or someone else's cubicle, or plenty of web-surfing.

The point is that the tester is blocked.

These blockages don't just slow down the project; they turn the tester into a very expensive paperweight.

So I want to get into the ranks and find out what is really going on.

Some of the time, the problem is systemic. The organization doesn't even realize that many of its people are sitting around, waiting for work. When it happens, people often feel they need to hide these moments. After all, they are embarrassing and, to the hatchet man, being not busy indicates redundancy, and that can lead to layoffs.

As a manager, I want to find the systemic blockage and fix it. I will work with the tester to figure out either how to knock down the obstacle or to bring the problem to the organization’s attention so it is not mislabeled a test cost. Another option may be to redirect the tester to work on something more actionable.

One of the biggest conclusions I came to when researching the book was this idea that "three strikes and you are not out!" There is always something a good tester can do to influence the outcome of the project. Even if system forces are keeping the testers down, it is usually possible to make some progress with a strong, can-do culture.

The second aspect of culture is that some folks might not be blocked, but just don't want to work that hard. If you do management by walking around, say, two trips around the office every day, and see the same people not working ever, and find out they are not blocked, you've just identified your low performers. You can take any traditional corrective action, but if it were me, I'd start with the managers. How did this situation happen?

Tip two: Identify and remove barriers to high performance

Once it's possible for our testers to actually make progress, we'll take a look at how that time breaks down. Are they actually doing testing, running experiments to see if the software will work, or are they doing other things? Going to meetings, writing documentation, setting up test environments, loading databases with test data so the tests can execute, these things are incidental. They are not essential to the task at hand.

If possible, I'd like the testers to write down how they spend their time for a week or two, to break the work down into major categories. Examples include testing, setup, documentation, meetings and administrative tasks.

If the testers are spending a great deal of time in documentation, one thing they are doing is not testing. This is also true for setup and meetings.

It might not be a good idea to eliminate meetings, documentation, and setup -- but it is not uncommon to find a team spending 80% of its time doing other things besides testing. Get that down to 60% -- just take a small chunk out of it -- and you could double productivity without having to "adopt" a methodology, buy a tool or send anyone to training.

The trick is to see those things as waste and eliminate them. That may take some willpower, some decisiveness and some ingenuity.

I said you could do it. I never said it would be easy.

Step three: Speed the test process

Speaking of waste, let's take a look at that 20% of time testers are "actually" testing. What are they doing?

They may be trying to reproduce a bug that is hard to nail down, or exploring around the issue, trying to characterize it. Perhaps the tester is writing a bug report, or attending a triage meeting.  

Notice: None of those things is essential for software testing -- they only exist because bugs exist. For that matter, because they take time, doing those activities subtract from time spent actually testing.

Forget 20%. Some teams I've worked around have spent as little as 5% of their time making actual forward progress testing -- the rest of that time is spent on these incidental things that only exist because a bug was there in the first place.

We may not be able to eliminate bugs, but in many cases, the software could be a whole lot better before it gets to test, and that means testers can spend more time testing, less communicating about issues and waiting for builds.

At this point, I want to look at how that testing time is spent, and ask if we can get builds of better quality before it gets to test.

There are lots of ways to do this. One way is to simply present the low quality initial builds to the whole team as an engineering problem, and ask them to come up with the corrective actions to fix.

Step four: Eliminate excess work-in-progress inventory

In physical manufacturing, people understand that excess work-in-progress (WIP) inventory is a sort of waste; it's money put into the system that will not have value, and, in some cases, could lead to discounted sales, clearances and so on. Once products hit the five-and-dime, the manufacturer is probably losing money on every part -- because it made products in huge batches beyond what the market could bear, instead of selling a dribble at a time.

It's the same thing with software. Excess work-in-progress can lead to waste.

There are entire methodologies around applying lean to software testing; we've written about some of them right here at SearchSoftwareQuality.com.

Once your team is performing, you may want to reduce work in progress and pursue one-piece flow. It's challenging, can change the way you think about software testing... and it's totally worth it.


This method is not easy and it is not instant. One claim I can make in good faith: I have worked with companies that applied these principles, and have moved from a test bottleneck to an abundance of test/qa time and energy. Teams I've worked with have had enough abundance to hire more developers without hiring more testers, or to move some testers on to other teams, or to expand the responsibilities of the test discipline to include more product quality issues.

We go into much, much more depth in the contributed work I edited on this very same subject: How To Reduce the Cost of Software Testing, but I’ve hit upon the highlights of four quick tips that are sure to give you the biggest bang for your testing buck.

What suggestions do you have for ways to reduce testing costs in your organization, and still retain quality? Let us know by sending an email to [email protected].

Dig Deeper on Topics Archive