Test-driven development (TDD) may be "kind of the Holy Grail," but "the journey is almost as worthy as the destination," according to Dave West, a senior analyst at Forrester Research in Cambridge, Mass.
It's a journey that organizations transitioning to agile often struggle the most with, out of all the agile practices.
That's true at IBM Software Group, based in Midvale, Utah, which started its agile transformation in 2007. According to Sue McKinney, vice president of IBM's development transformation, test-driven development is encouraged, but "it's probably the one thing we're struggling the most with."
"We do a lot of unit and function test automation in general, and we're getting better with integration testing, but it's a mindset. Developers like to develop code; they don't necessarily like to test," McKinney said.
But it's a journey they're going to continue, she added. "We have folks from acquisitions who are architects that are exemplars of good approaches to TDD. We had them write papers and socialize how they got it started. In 2009 this will be one of the focus areas, to get development teams thinking about TDD."
TDD has its roots in Extreme Programming (XP) and traditionally refers to the practice of developers creating automated unit tests that define code requirements before writing the code itself. Kirk Knoernschild, an analyst at Burton Group, said TDD is sometimes called test-driven design. Knoernschild noted some of the challenges of the process: "If the units become too large it's hard to write high-quality tests." Also, he added, "a lot of organizations say developers are writing unit tests, but I don't know what percent code coverage they're achieving. And a lot of those same organizations still continue to delay other forms of test to late in the software lifecycle, giving all [the problems] time to fester."
Legacy code can also create issues. According to Chris Kinsman, vice president and chief architect at AMS Services, a provider of insurance agency automation products in Bothell, Wash., AMS has stepped up the amount of unit and integration tests written around the code base, but they're not doing a lot of TDD as part of their agile efforts.
"We struggle with a large code base not written using TDD. It's difficult to … be test-driven in a legacy code base," he said. "Some teams have tackled [TDD] around a sprint, but it's not really a practice we've adopted across teams."
He said he's not sure if he'll make TDD a goal. "I have concerns if it's doable in a code base not designed to work that way. We have some architectural issues before we get there. I'm not philosophically opposed; I just don't want to make our jobs harder," Kinsman said.
Another challenge of test-driven development is the potential perception that it's slowing the development team down, Knoernschild said. A good analogy, he said, is the practice of double-entry bookkeeping.
"It would be much faster for the accountant if he didn't practice double-entry bookkeeping, but at end of day if you're off by a certain amount you won't have the checks and balances in place to find the error [easily]," he said. "It's the exact same way with testing. If problems arise you don't have a system of checks and balances in place to isolate and identify the exact nature of the problem."
While TDD can slow things down initially, he said, "as the code base grows … it will speed things up" in the long term.
It's a cultural change, said Forrester's West. "As an ex-software engineer, when you're in doubt you write a lot, but a lot of early documents avoid saying what it is by saying lots of things. The discipline that allows you to challenge the problem up front isn't there; you're not encouraged to do that."
Also, West added, sometimes the developer may need to explore a bit before he or she knows enough to write a test. "It isn't that you don't want to be prescriptive, but until you write some code, sketch a few things out, you don't understand the problem well enough to build a test. In the process of experimentation, which agile encourages, you understand the problem better, which allows you be more prescriptive."
The departmentalization of testing resources can also be a hurdle, West said. "Some vendors can be to blame for this; they encouraged centers of excellence -- departments focused for QA." This can make it difficult to get QA involved early, "so you do agile without them," he said. "You do unit testing, but you do everything else avoiding them because they're a separate department."
At Vignette, a Web content management company in Austin, Texas, some teams are starting to embrace TDD, but it's not yet widespread, said Subu Subramanian, senior director of engineering. "When some teams start thinking about user stories and acceptance criteria, they elaborate it in the form of a test, so while design and development is going on they're thinking about tests."
However, ultimately the goal at Vignette is to bring testing forward earlier in the cycle, Subramanian said.
And that is often the benefit of a foray into test-driven development, even if it's not part of an agile effort, Knoernschild said. "TDD is often associated with agile development, and it's certainly an agile practice, but even if you're not going full-hog into agile the value of TDD is still significant," he said. "From the minute you write the first line of code you can also be moving tests earlier into the lifecycle. You want to grow the test suite right alongside the growth of the source code."
West added, "How you implement and approach [TDD] depends on your organization. You have to be pragmatic. Maybe it's implementing a unit-driven approach that allows two people working, one doing test and one doing code."
However the journey to TDD goes, West said, organizations need to "have a maniacal focus on quality and test data at an early stage."