VSTS and testing today, Part 3 - Are your tests test-driven?

Test-driven development (TDD) is becoming more popular with developers. But is unit testing truly compatible with TDD? Can the VSTS tool improve the process?

Although definitions can be tricky, unit testing has been pretty intrinsic to Extreme Programming (XP) methods, as well as agile development styles that have come along subsequently. While unit testing in and of itself can help drive a better development process, some believe it does not necessarily support an agile test-driven development (TDD) process.

But test tools such as csUnit, MbUnit and TestRunner, some claim, can have special use for agile development of VS applications. Meanwhile, some of these same individuals question whether Visual Studio Team System adequately supports a true test-driven development process.

Certainly, unit testing is part of TDD, but some contend TDD is an important extension to XP, Agile methods and unit testing. If we ask Mr. Wikipedia what TDD is, we get the answer: "Test-Driven Development (TDD) is a computer programming technique that involves repeatedly first writing a test case and then implementing only the code necessary to pass the test. The goal of test-driven development is to achieve rapid feedback. The technique began to receive publicity in the early 2000s as an aspect of Extreme Programming, but more recently is creating more general interest in its own right."

Manfred Lange, one of the csUnit Lead Developers, explained, "There are a lot of people who believe that unit testing is one of the agile practices (XP in particular), and that when unit testing is used, a team has already adopted a first agile technique. Unfortunately, this is not quite true."

"There are two battles going on and most people only see one. One is getting developers to practice unit testing and two is getting them to practice test-driven development (TDD)," said Jay Flowers, a Design Lead at Northrop Grumman.

"There is a lot of legacy code out there -- legacy code as defined by [noted programmer and consultant] Michael Feathers as 'any code without unit tests.'

"I think the key to winning the first battle is to make it easy to get developers' legacy code under test. Getting their legacy code under test, and under test easily, will flow into adoption of TDD," said Flowers. "Getting legacy code under test is a technical issue where our adoption of TDD is a behavioral change."

"A success in the first battle will give momentum to attacking the second."

Test first, ask questions later

In TDD the test always comes first, while in unit testing this is not always the case. From this perspective, TDD programmers are not able to use the automated test generation features available in VSTS, according to some.

Manfred Lange said, "The basic idea of TDD is: Test a little, code a little, refactor a little - in that sequence! And the rhythm here is not in hours or days."

"The rhythm for the entire sequence is seconds or minutes. Run tests frequently, preferably as often as possible. With TDD the test always comes first, while with 'unit testing' in the broadest sense this is not clear. Generating tests after the fact is not considered to be TDD. Sometimes I even prefer the term 'test-driven design' as it makes clearer what this really is about."

On the other hand, unit testing after the code is developed is not necessarily bad, it just does not serve the goals of TDD. Lange explained, "Some testing is always better than no testing. However, software that is implemented using a test-driven approach, if done properly, will result in simpler systems, which are easier to understand, easier to maintain and with a near-zero-defect quality. The design will look different and, if the developer has sufficient courage, even the architecture can emerge over time using this approach (instead of up-front architecture or up-front design)."

Automation downside?

The main benefit of the automated test generation features in VSTS is to help get the legacy code under test, noted Flowers. Although, he suggested it is not a very smart code generator and is writing a better one called Doubler.

Will Ballard, Owner of Mailframe, which makes the TestRunner tool for VS, said, "With VSTS, the unit testing tools are most useful in running a large battery of tests, and not well suited to single test case iteration in TDD. I like that it is integrated, but I dislike the 'list view' style visualizations in VSTS: Too much scrolling and shuffling as you get a real-size set of tests in an assembly, which for me are often in excess of 1,000 test cases."

In contrast, other tools might be more suited for TDD. "With TestRunner," claimed Ballard, "the focus is on the in-editor experience that is central to test-driven or test-first development, where the coding takes place as a continuous test-code-test sequence of activities.

Unit testing explored:
VSTS amd testing today, Part 1 - Revenge of the unit 

VSTS and testing today, Part 2 - Capabilities and plug-ins 

Parasoft C++ extends automated unit testing to embedded systems environment

"You can run from the 'test tree' view, but you can get at all the test functionality right there in place as you type a test case or change your application code. Integrating profiling and code coverage with unit testing was the real innovation, as you can see how well your tests are exercising your code. I found that folks using external coverage tools simply gave up on them after extended use, as it became a cumbersome additional step."

Another testing framework is MbUnit, which Flowers has been working on. He said it is easy to extend, has more types of test fixtures, and supports an add-in called TestDriven.NET.

He noted, "TestDriven.NET can run test written with just about any framework. So in the end the only reason to buy VSTS is to enforce your process."

"These other tools do a better job of encouraging users to follow the process than VSTS does. In the end I see MbUnit, Doubler, TestDriven.NET, nCover, NAnt and CCNET as a better toolset than VSTS, and this toolset is free and will continue to evolve and release new features at a much faster rate than VSTS can," he said.

George Lawton is a technical writer based in Brisbane, CA. He has written for Software Magazine, Application Development Trends, IEEE Software and other technology publications.

This article originally appeared on TheServerSide.NET.

Dig deeper on Software Security Testing and Quality Assurance



Enjoy the benefits of Pro+ membership, learn more and join.



Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: