I’m here at the Software Test Professionals Conference in Las Vegas, Nevada. Yesterday, I had the opportunity to participate in a panel discussion on “How to Reduce the Cost of Testing.” It was fun.
Cutting the costs of testing
So here’s the problem: American business has a mandate for growth in profits. They can accomplish this in two ways: grow revenues or cut costs. Looking at the testing group as a cost center gets you one choice: cut costs.
A small group of passionate testers is currently working on a book on the subject, and several chapter contributors happened to be at the conference. So we got the group together for a panel discussion to cover what ideas they had to reduce the cost of testing with integrity. Please allow me to share some of their answers:
Justin Hunter suggested that most bugs come from combining two or more inputs. So we can collapse our tests into only those tests that actually cover pairs of test conditions.
Scott Barber wanted to talk to executives about the other side of the equation – the social value of having software that actually works.
Catherine Powell was interested in the “great game of testing.” Given that we have limited time and budget and an infintite number of test ideas, what should we be testing right now in the time we have. The formal tool she suggested to compare ideas is designed to measure the opportunity cost of various test strategies.
Selena Delesie talked about the cost of quality, and how to discuss it with executives.
Lanette Creamersuggested collaborative testing – basically decreasing the organizational boundaries and the cost of tradeoffs between business people, developers and testers, through pair testing, story kick-off and so on.
Finally, Petteri Lyytinen (yes, he’s Finnish) had a small set of ideas like Test Driven Development and Continuous Integration to try which would allow teams to decrease the length of a test cycle.
Then what? Lay off your staff?
As moderator, I got to ask the leading question: Let’s say you are successful at reducing the cost of testing, and can now do with eight people what used to take ten. Does that mean that you have just earned the dubious privilege of laying off your staff?
Lanette expressed that senior management was likely going to cut anyway, so this was advice on how to deal with those cuts. Several other people stated that these ideas were designed to meet management where they are at, then move the discussion to include value. (Scott Barber made it clear that if you can’t quantify the value, you can’t decide if any cost is “too high,” “about right,” or a good deal.)
How about just preventing the bugs in the first place?
Then the audience got to ask their questions. My personal favorite was the person asking about traditional manufacturing ideas that we should “just” prevent “all” bugs “up front.” The general consensus was that, while those ideas have some things to offer, the analogy is not correct: With manufacturing, you do need to test one time, at least, for every change in process.
The thing about software is that every check-in of new code combined with a new build, is a change in your build process.
So one place to start cutting the cost of software is not eliminating practice X, process Y or person Z. It’s in actually understanding why those practices and processes are in place, then deciding if the risk is worth not worrying about.
Do you have ideas of your own? Let us know!