Tip

Code coverage: Beyond the basics

In the first of this two-part series on code coverage, we briefly looked at how code coverage works with a specific example. We found that while code coverage tools may measure code that has been

Requires Free Membership to View

executed with unit tests, it doesn’t necessarily validate the usefulness of those tests.  In this second tip, we look more deeply at the different types of code coverage metrics and end with recommendations on how to best increase code coverage and where to go for additional information.

Different types of coverage

Most code coverage tools provide different levels of coverage. There are three broad categories of coverage: statement (or line) coverage, branch coverage and path coverage. We saw an example of statement coverage in the example above, but let's look at branch coverage and path coverage in a bit more detail.

Looking at the example code in Listing 2 below, suppose you had a unit test and passed in the value 1 for both the row and column parameters and as a pre-condition set that square on the board to be null.

def isTargetBlank?(row, column)

   if @board[row][column].getSpaceValue() == nil then

      return true

   else

      return false

   end

end

 Listing 2: Example method for evaluating an empty space on the board in Gobblet.

In that case, with statement coverage you'd exercise the if statement line, return true line, and the end lines. Most statement coverage tools would likely give you five lines out of seven. I say most tools, because some tools can calculate coverage a bit differently.

However, with branch coverage we're more concerned with the evaluation of control structures (like the if statement).  So in our example above, we'd likely get 50% branch coverage for that method. That's because we've tested one of the two possible evaluations of the state of the space (it being null or not null).

With path coverage we're looking to evaluate if every possible route through the code has been executed. For this example, our branch coverage and path coverage will match. Because we have an if statement (and only one if statement), so we also have 50% path coverage. There are only two paths through the code, the if condition or the else condition.

If we change the example to something like what's shown in Listing 3, it gets a bit more complicated (but still not too complicated).

def removePiece(row, column)

   if (row < 0) or (row > 3) then

      setMessage("Row not valid: " + row.to_s)

      return false

   end

   if (column < 0) or (column > 3) then

      setMessage("Column not valid: " + column.to_s)

      return false

   end

   @board[row][column].pop()

   self.setMessage(nil)

   return true

end

 Listing 3: Example method for removing a game piece from the board in Gobblet.

In this example, if you wanted 100% branch coverage, you'd need at least five tests. You'll need one for each side of the or condition in each of the if statements. That's four tests. And you'll need one that successfully "passes" each if statement to execute the return true statement. For 100% path coverage however, you'll only need three tests - one for each of the two error checks (returning false) and one that successfully removes a piece. Once again, those numbers may depend on which tool you're using and how they've implemented their coverage algorithm.

It's been my experience that most teams measuring coverage focus on getting high statement coverage with reasonably high branch coverage when possible. I've not worked with a team that's given much attention to path coverage. As near as I can tell, for code of any complexity, it's likely more trouble than it's worth.

Tips for increasing code coverage

When you start testing for a particular piece of code, don't think about coverage at all. Just write the tests you feel are necessary to implement the functionality or behavior that you're looking to get from the code. Focus on good testing, not an arbitrary coverage metric. As you write your tests, you want to be thinking of clean interface design, test data, and mocks that support your testing.

Once you feel you have the tests you need, then run the coverage numbers. Use that information to help figure out what you might have missed with your earlier testing. Is there a condition you missed that you feel you should have a test for? Go ahead and implement it.

Keep in mind that at this point if you feel your tests cover all the functionality, code that's not exercised might indicate code that's no longer needed. Double check. One way to increase coverage is to reduce the footprint of the code base by removing code that's no longer needed and not exercised.

When you're trying to increase path coverage, many times you might have to refactor the code into more testable methods. High cyclomatic complexity translates into a lot of paths to test. If you can lower that complexity, often times you'll find it easier to increase coverage.

Next steps

There are a number of coverage tools listed out on the Wikipedia page for code coverage. If you're looking for a tool, that's a good place to start. More important than the focus on tools, I’d recommend some articles on code coverage that might help make sure you’re focusing on the right aspects:

For more on measuring quality, see Quality metrics: A guide to measuring software quality.

 

This was first published in November 2011

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.