WavebreakmediaMicro - Fotolia

Manage Learn to apply best practices and optimize your operations.

The secrets of successful Lean software testing

Are you overwhelmed by Lean software testing options? Expert Matthew Heusser explains in detail how to get your team on the Lean bandwagon.

There are quite a few articles about Lean concepts for software quality, including ideas such as continual improvement, decreasing the size of the work and watching the flow.

It's easy enough to apply these to a work process or to an individual. Today, we'll explore what "Lean teamwork" might mean -- and how to get there.

On teamwork -- and competition

The first issue with creating Lean software testing teams is actually having teams. Many organizations structure teams by specialty, with the testers in one group, programmers in another, operations in a third, then the analyst group, the project management team and so on. A portfolio management group picks a handful of people from each team to form a project team that exists for a short time and then gets disbanded. The result is the roles are set up in opposition to each other -- testing is pitted against the programmers and analysts. The managers of the different specialties also likely compete against each other for promotions and attention. This is especially true when the company is using a forced stack ranking, which means the test team members are also competing with each other. Competition can be good, but when someone else winning means you lose, it can drive the wrong behaviors. For example, if a staff member knows a shortcut to get work done faster, sharing that information causes them to lose an edge at evaluation time. The result is hoarding and controlling of information, which hurts team performance.

Step one: Organize as an actual team. Scrum, Extreme Programming and most Agile processes require an integrated team that has all the skills and tools to deliver software from proposal to production. This is compatible with some Lean ideas, such as flow -- requesting outside resources to deploy to production, for example, causes a delay and a handoff. More importantly, it creates the right behaviors. Driving out destructive competition might not be a Lean principle, but it isn't very hard to derive from the principles.

That includes performance metrics. When individuals are measured and those measures go up on a wall, it creates the appearance of competition. At the very least, some Lean software testing team members may be reluctant to help each other, as "I need to get my work done first." These sorts of measures are hard to do when the team is multiskilled and self-organizing. That's good, because you don't need to measure them at all. Instead, look at the team performance, find the bottleneck and address it. Everything else is suboptimization -- improving the strength of anything that is not the weakest link in a chain is a waste of time.

Once you've analyzed for team performance, establish a direction for improvement. Predictability, innovation, low defects and faster time to market are all valid directions for improvement -- and they might conflict with each other. Without a defined direction, improvement boils down to whose idea is the loudest at the moment. With a direction in place, the Lean software testing team can create a target condition together. That's a next step that seems achievable, though challenging. For example, the team might want to get to faster velocity long term (the direction) by reducing defects (target condition). One target condition might have developers and testers spend no more than 20% of their time documenting defects and doing rework -- although measuring that might be challenging. Another might be no more than one interruption for bugs per day, which is easier to count. These measures are shared by the team, do not judge the person and are thrown away when the condition is achieved, which should minimize game-the-metrics behavior.

In other words, the team needs to really believe they are a team, that the measures are for their own improvement, are incredibly easy to gather and will not be used against them.

You'll notice that one bug interruption per day is the desired outcome. What is left undefined is the operating attributes the team will have -- the how -- to accomplish the target. Those can be defined as goals, too, and they might change. For example, the team might suggest peer review would reduce the defect count, but finds that it doesn't work to identify defects that matter, and then reject code review in favor of something else.

Lean teams -- and Lean management

In Lean manufacturing, one typical practice is to have an andon, a cord tied to a light. When a worker needs help, he pulls the andon and a light appears over his head. A supervisor sees the light, walks over and addresses the issue, so the worker can continue to work and the flow of work is not interrupted. The next level of supervisor looks for patterns of cord pulls to change the process to resolve or prevent problems. This means the first line supervisor is a participating member of the team, not an adversary or judge.

Mike Rother's book, Toyota Kata, goes on to describe a situation where the plant was staffed for 1,000 andon pulls a month and had gone down to 700. A typical corporate response would be, "No problems means no problem." Instead, the plant manager called an all-hands meeting and pointed out two possibilities: Real problems were emerging and team members were not calling for help, which slows down the line, or else inventory levels were so high that there were no problems. He encouraged people to report real problems, and, if they didn't, informed them they would be slowly drawing down inventory levels until the problems resurfaced. In software delivery, this could happen when schedules are always 100% on time -- too much schedule buffer -- or there are not enough bugs; either they are going unreported, or the team might be going too slowly.

That advice might not apply in your situation, but it is certainly a different way of thinking about improvement and improving speed. More importantly, it encourages real Lean software testing teamwork over hiding problems to look better.

Making the change

The idea of Lean teams that I introduced above runs directly contrary to the culture in many organizations. In addition to dealing with resistant co-workers, the very idea of just getting started can be overwhelming.

So, start where you are. Lean software testing project teams can frame the idea of target conditions as risk management -- listing the risks on the project, then tackling the No. 1 risk until it isn't No. 1 anymore and repeating. The self-organized, multidisciplinary nature of the teams listed here is very compatible with Agile transitions. The retrospective format -- where you gather the whole team to discuss how things worked out -- is a great place to talk about improvement direction and target conditions.

The retrospective can also be a great place to change culture by example. One team I worked near had retrospective notes that listed the deficiencies in the performance of each person. Instead of blaming behavior, look at the system forces driving the behavior: Are we not providing training? Are we hiring people without skill or interest?

Changing the tone of conversation in the moment is the kind of thing anyone can do. It is that kind of behavior that creates a culture, and even defines it.

When you can't change

Large, hierarchal organizations with defined policies are unlikely to make these changes on a request. In that case, you might be better off to work around them. If you can't get rid of annual competitive performance reviews, you can follow the advice of Joel Spolsky, the CEO of Fog Creek Software, and just give everyone a gushing review. If you are not allowed to give an "exceeds expectations" to everyone for whatever reason, give them all "average" and explain why. View the role of management as protecting the Lean software testing team while looking for patterns and resolving them. You might not be able to make the whole company Lean, but if you can make a project or team high-performing for an extended period of time, people will take notice.

Next Steps

How big companies can go lean

Testing on a budget -- it can be done

Should testers learn how to code?

What advice should you expect from a software testing mentor?

Dig Deeper on Software testing tools and techniques

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Is your enterprise taking steps towards Lean software testing?
As I commented in the parallel thread, that's a good article and I learned from it.

Now here I'd like to point at a few potential "risks" of adopting it.
  • Since everyone is brought into a single team, obviously, the organization won't need that many managers. That can't be good.
  • Once people learn to truly work together they may realize that their team becomes a power on their own. They become uncontrollable!
  • They may even learn to take care of their problems internally, and negotiating with external contacts as a single. Liaison staff might become not needed.

Kind of scary, isn't it?

We are working towards Lean software testing. One truly beneficial change that’s just happened company wide was removing annual performance appraisals. I think that’s a key step, and is sure to help us establish the true team that Matt talks about.
It is difficult to say what that would look like in large enterprises.  There is always this push to 'standardize' and I have a feeling Lean software delivery and testing may look a little different for teams working on different products.
Many ideas of this article are very consistent with the ideas expressed by Gerald Weinberg's book "Managing Teams Congruently".

Another couple of ideas from the book that should fit well the "Lean Testing" concept.
  • "If you feel you have to give performance appraisals, a more effective way is to appraise the team performance and ask the team to appraise the performance of each member. The external performance rating is obtained by multiplying the team's overall rating by the individual's rating as given by the team. This approach says you trust the team to do what's best for meeting the goals you give it, and that they all get credit equally from you for meeting those goals as a team."
  • "One of the easiest places to see the impact of teamwork on quality is the fault location team. These teams are trained to work together on fault location and to prevent the problem of failure reports circulating endlessly and failing to reach the person who can solve them. To create an effective fault location team, the first step is selecting a diverse set of members. Diversity is the key to effective fault location teams—diversity of experience, of organizational connections, but especially of thought patterns. If your thought pattern differs from mine, most of the errors I miss will not be the same as the errors you miss."
Culture has a lot to do with how we implement something like Lean Software Testing. Many times, I see an idea not implemented because it wasn’t a good fit with corporate culture. I think the main thing to remember is that a good idea shouldn’t be abandoned because it’s not a good fit. Take that idea, rework it so that it is a better fit, and try again.
I cannot say enough about how bad metrics create dysfunction.  Sometimes even the good ones, when given too much emphasis can drive bad behaviors in teams.  This applies to lean as well as any other methodology.