After the Healthcare.gov fiasco, it's become apparent that Agile projects need constant maintenance to stay healthy. Performance metrics are a useful method to determine if an initiative is on track or headed off the rails. There are different kinds of project metrics an Agile team can use to monitor performance
Project progress metrics
The zeroth metrics are those that should come for free with an Agile project. One of these is velocity -- the rate at which you complete each iteration.
With velocity, you measure which day of the iteration you complete the feature. The chart above (Figure 1) is an illustration of a successful iteration, a useful way to visually confirm that every day of the iteration is moving the project forward.
That's fine for each team. But that doesn't reveal much about the project as a whole. What you really need to see is velocity over time.
This velocity, expressed in a burn-up chart (Figure 2), is much more interesting. It shows how many points a team completed and how much the points increased over the course of the project. Have you ever met a project that didn't have more requirements added over time? Figure 2 shows that variable.
If you only track a project's progress iteration by iteration, you see a snapshot in time. With the burn-up chart, however, you see the larger trend of the project over time. It can reveal many potential problems in your project. If the team doesn't finish features consistently, at the end of each iteration, you won't see a nice upward slope to that blue line that says "Features Done."
If the product owner adds more features faster than the team can finish features, this chart will show that. You can see that this product owner added features at Iteration 4. The team maintained its pace. The product owner again added features at Iteration 7. The team still maintained its pace. If the product owner had added features at Iteration 11, the team would have been able to finish the project, but not at Iteration 12. The team would have needed more time. A burn-up chart such as this
one allows you to predict better than an estimate does, if your product owner has practice at creating small-enough stories. Otherwise, you have to estimate.
How many defects are you creating?
I like to track defects in an Agile project because I want to expose and eliminate them. That way, we can stamp them out before they create technical debt. When left untended, technical debt exacerbates problems, making them more and more difficult to solve in the future. Sure, there are times we will choose to have technical debt. But, if we are conscious about these choices, we can minimize having to make them.
Figure 4: Defect Trends Chart. This chart shows the number of newly found defects, open defects and closed defects.
One chart is the iteration contents chart (Figure 3). This tells you what is in each iteration. It's particularly useful for retrospectives. As Figure 3 demonstrates, when the team shifted from trying to do features to accommodating changes in their iteration, they created more defects. I've seen that happen in other teams, as well. Maybe it's even happened to you.
Again, we want to track defects as trends over the life of the project. What happens in an iteration is interesting, but not as interesting as what occurs over the entire project. I use two different charts to track defects.
This is a normal defect trend chart that could be from any project. You want to track the number of newly found defects and closed defects. The trend that's particularly interesting here is the number of remaining open defects.
In an Agile project, you would expect that number to be very close to zero. If it's not, you have a Healthcare.gov situation. But at least you would know about it.
Seeing the total number of features in process
One thing I like to track is the total number of features in process. The fewer number of features in process, the more work you have completed. When there are more features in process, an Agile project has fallen victim to the multitasking problem.
If you focus everyone's effort on just one feature and swarm around it, you get the feature done fast. If everyone takes their own feature, it takes a lot longer.
As it turns out, this can be measured. The average time it takes to get a feature to done for a team is called cycle time.
If your cycle time is one or two days, you might not care about that. That's why I like small stories. To see what your cumulative flow is, try a chart like this.
You can use a cumulative flow chart for any project. If you use it for a Waterfall project, you start everything at the beginning of the project, so you have lots of red and yellow on your chart and not too much green. As you complete features, the yellow starts to vanish and the green takes over.
Now you're done. Start with these project metrics, add or subtract as the project proceeds, and you will have a much better understanding of what your Agile team is doing right and what it needs to change.
This was first published in December 2013