It is time to tackle the common question: "Why is QA always the bottleneck?"
In a previous article, I introduced the concept of the unfair question when examining questions commonly thrown at quality assurance (QA) and testing pros. Generally, such questions are not questions at all. Take, for example, "Why are you lying to me?" That's more accusation than question. Attempting to answer it in any way is a defense; in fact, it is a confession. So, caution is absolutely necessary.
Read this first.
I've made some assumptions to keep these articles short, and I'd like to clarify them. First of all, they assume the question – Why is QA always the bottleneck? -- is asked in a dysfunctional way, which is a judgment decision. The answers I provide are designed to change the discussion, to "jiggle" things back into healthy shape.
You don't need the answers if, say, your manager asks the question and offers to discuss how to prevent the problem in the future over company-sponsored beer. Secondly, they are not a quick fix or a manipulation tool. While they could be used that way, my goal, and I hope yours as well, is to educate and collaborate, not one-up.
So, I hope this can help if you're asked an accusatory question in an accusatory way and are searching for a way to respond that can break the ice.
"Why is QA always the bottleneck?"
In testing we take some stuff that may or may not be fit for use and – STOP – before we deploy it, do some checks. Those checks often find problems. Our whole goal is to make sure things are in some reasonable level of goodness before the software is shipped. This is even truer in organizations that throw around the term QA and expect the group to actually assure quality.
So yes, we can come up with test design ideas early. We may even be able to automate some accept...
To continue reading for free, register below or login
To read more you must become a member of SearchSoftwareQuality.com
ance tests in parallel with the code. However, if we want any of our testing to learn or adapt to the software, in one way or another, we're going to be the bottleneck.
Armed with that information, we can always try to do our job better to reduce the time from 'code complete' to ship. The key is, we won't feel bad about being the bottleneck. Moving beyond shame, we can actually work to improve the process by things other than working harder and wishful thinking.
Let's look at some of our tools from my unfair questions tip:
Embrace the premise
Let's take the idea that QA is the bottleneck on face value. If that's true, management is asking us how we can go faster. Well, what is slowing us down? Here's a short list, all of which I have seen working in industry:
- We are stuck in fifteen hours of meetings a week.
- We are required to create or update documents we see no value in.
- It takes hours to setup and tear down test environments.
- I have problems reproducing defects, and the developers are in another building. If we could just sit together for a couple hours a day, I am certain we would knock this out quickly.
- A great deal of our work is massively repetitive; we could decrease overall testing time if we had some way automating this work.
- Some of these tests just never fail; we aren't learning anything when we run them, and even if they broke, it would only be a hindrance, not a blocker.
- I need to poke the GUI a hundred times for a test run. I would really like to feed in a data file to app and have it loaded for us.
In other words, this could be chance to eliminate waste in the test process or for management to invest in testability for the application or test tools to speed up testing.
Reject the premise
When you're testing, what are the developers doing? It's possible they are sitting idle or that they are working on other projects, but it's also likely they are fixing bugs.
At one local tester's discussion, Michael Bolton once pointed out to me that the moment you find a blocker bug, it's not a testing phase, it's a fixing phase. In this case, testing isn't blocking release, development is blocking releasing.
If management wants to know when they can release, they will likely be better off asking the developers when the bugs will be fixed. In that light, testing may only be a bottleneck by a number of hours, days at the most.
So, what's the answer to the question, "Why are we blocked by testing?" It might just be: "Last time I checked the blocker bugs list, QA wasn't the bottleneck, bug fixing was. Has that changed?"
Expand the question
It's likely the question isn't really about optimizing testing; it's about optimizing release. Even limited to testing, we can reframe it as, say, "What is slowing down testing, and what can we do to speed it up?"
Test manager and consultant Jonathan Bach once suggested the following exercise. Imagine a test team that never found any defects. They would sail right through the app in a few hours or days and say done, right? Compare that to a buggy app. Each time the team finds a bug, they need to stop, find a reproduction case, document the case, then test it again tomorrow. Each of these stops is extra work that switches our focus. The first team in the example is going to finish first; perhaps days or weeks earlier.
The moral is that one way for the test team to go faster would be to have higher quality before the code is turned over to the team.
Engaging management in problem-solving
It's no secret that management wants testing to be done faster. What they might not know is exactly what work is involved in the test – what risks we see, how we plan on attacking the software, and how much time we plan to spend on each feature. So we could take the tac of laying out all the work that needs to be done in a nice line and asking management "what can we skip?"
It is possible that having confidence in some nooks and corners of the software is less important than shipping a day or two early. Further, it is possible that management may authorize certain risks (skipped tests) in order to hit the date. If they do, testing can go much faster.
And, when those portions of the app are buggy as all get-out, the same executives will sagely nod and agree "well, that was a calculated risk we took" because they have joint ownership in the decision. Now, that's more than a get out of jail free card; it means that next time you make the case for extended testing, you'll be able to say "let's not let this turn into another SquareCalc project." You might even get the extra time you ask for.
If the best possible outcome is to be able to help out the next project, well, it's still the best possible outcome. You may do well to take it.
Use Humor
Now, I'm all for committing to a project and working hard to have it be successful. AND there comes a point where some invisible line is crossed, where "aggressive" becomes "unreasonable." When I was in that situation, this is how I replied:
"You know, my father used to joke with the cashiers at the grocery store 'twelve dollars? I don't know …' When I was sixteen and got a car and drove to the store, I tried it too. But my sarcasm wasn't too good, and the clerk took me seriously! 'Well sir', she said 'if you want to pay less, you can take something out of the bag … or you can shop somewhere else.'" At this point, the team broke out laughing, realizing that's what the company was asking -- more software for less schedule and budget.
Now I could never have challenged the manager in a debate about the schedule, but that single anecdote deflated the entire conversation.
Challenge = opportunity
There are multiple interpretations of "Why is QA always the bottleneck?" One is the general idea that senior management wants the software done faster and is whipping the horses. Another is that senior management really doesn't understand, or hasn't given deep thought to, exactly what is going on in during test activities.
In either case, when we try to interpret and respond to things in the most positive and healthy way, we have a chance to educate, create trust and possibly even change the rules of the game.
"Why is QA always the bottleneck" might be an accusation, but it also might be the magic moment that changes the way the team thinks about its test function. Ultimately, all we can control is how we respond. Let's respond well.
About the authors: Markus Gaertner is a software tester for Orga Systems in Paderborn, Germany. Matt Heusser is a technical staff member of SocialText, which he joined in 2008 after 11 years of developing, testing and/or managing software projects. He teaches informaton systems courses at Calvin College and is the original lead organizer of the Great Lakes Software Excellence Conference, now in it's fourth year. He writes about the dynamics of testing and development on his blog, Creative Chaos.
