QUESTION & ANSWER

Unit testing critical for improving software quality

By Michelle Davidson, Site Editor 25 Jul 2007 | SearchSoftwareQuality.com

Software quality news and advice Digg This!     StumbleUpon     Del.icio.us

Alberto Savoia, creator of The Way of Testivus

The Way of Testivus is very creative and even entertaining. What made you decide to create it?
Alberto Savoia: Thank you, I am glad you enjoyed it. The main motivation for writing The Way of Testivus was the desire to collect in one place the main ideas behind the Testivus philosophy of "Less unit testing dogma and more unit testing karma."

These days, almost everyone agrees that developers should play an active role in testing –- especially in writing and running unit tests on their own code. Unfortunately, there is still a huge chasm between knowing what should be done and actually doing it.

Based on what I have seen and heard in the past several years, I believe that a lot of developers have been scared off from unit testing by some of the more extreme developer testing practices, like test-driven development. While I believe that test-driven development is a brilliant idea and can work wonders, for a developer who has never had to write tests, the idea of writing tests before writing the code might be too big a first step.

Another problem is that when a software organization finally decides that developers should practice unit testing, in their enthusiasm they often go overboard and start with goals that are too ambitious or expect all developers to approach testing in the same way -- they are too ambitious and too dogmatic. This usually backfires. You need to give developers some time to get used to the idea and the freedom to approach unit testing in the way that works best for them -– at least at the beginning.

Until unit testing becomes a standard programmer practice, we have no chance of emerging from the so-called Software Crisis.

What has been the reaction to it? Are you hearing from programmers, testers or project managers?
Savoia: So far, most people who read it seem to like it. I have gotten a lot of positive feedback -– as well as suggestions and requests for more entries and topics. I have already seen translations in Spanish and French. One of the most common requests is to expand each Testivus epigram into a longer piece and to add some commentary and examples. I am working on a follow-on, tentatively called Testivus In Practice, to do just that.

Why is such importance being given to unit testing?
Savoia: Because I believe that until unit testing becomes a standard programmer practice, we have no chance of emerging from the so-called Software Crisis. Unit testing by itself will not solve all of our problems and ensure 100% bug-free code, nor will it prevent all schedule overruns, but it will definitely help. In my opinion there is no excuse for developers not testing their own code. It's the right and obvious thing to do. I feel funny when I am asked to justify unit testing by developers; it should be the other way around. People should justify why they are not practicing it.

What needs to be done to ensure unit testing is successful?
Savoia: For an individual developer, I believe that a key answer to this is question is contained in one of the most important epigrams in The Way of Testivus. Here's how I put it:

Think of code and test as one.

When writing the code, think of the test.
When writing the test, think of the code.

When you think of code and test as one,
testing is easy and code is beautiful."

In other words, don't approach testing as an afterthought. It's very difficult to test code that was not written with testability in mind.

At the team and group level, the most important factor in ensuring success is to avoid too much dogma and too many rules -- especially at the beginning. Too much dogma kills motivation and creativity. Here's how I put it in The Way of Testivus:

Don't get stuck on unit testing dogma

Dogma says:

"Do this.
Do only this.
Do it only this way.
And do it because I tell you."

Dogma is inflexible.
Testing needs flexibility.

Dogma kills creativity.
Testing needs creativity.

Another, very important thing is that since testing is a combinatorial problem, you have to use some automation or you will be inefficient and will spend time doing things that are best done by a computer:

Some tasks are best done by hand.
Other tasks are best done with tools.

It is not noble to do by hand what can be done better with a tool.
It is not wise to use your head when your head is not needed.

Read The Way of Testivus Thanks to the Seinfeld show, you've probably heard of Festivus. Now we have Testivus. The Way of Testivus is a unique booklet that drives home the importance of software testing, particularly unit testing by software developers.

What other types of testing should programmers be encouraged to do?
Savoia: At Agitar, in addition to unit testing, all developers have to participate in "bug hunts" before major releases. During a bug hunt, we all help QA to do some scenario, functional, performance and configuration testing. It gives you a real appreciation for how difficult it is to do QA and shows you the benefit of catching bugs earlier in the cycle. It's pretty embarrassing when a bug that could have easily been caught with a simple unit test ends up halting and restarting a QA cycle.

How have development methodologies such as Agile, Scrum and Extreme Programming (XP) affected programmers' roles in testing?
Savoia: If it weren't for XP, I don't believe that developer testing would be where it is today. XP and Agile are the cool methodologies these days. XP/Agile require developer testing, therefore developer testing is somewhat cool by association. It's safe to say that without XP and JUnit, we would not be where we are today. We still have a long way to go, but we have also made a lot of progress in the last several years.

In a perfect world, what would the software development life cycle look like so that companies create applications that are bug-free, meet user requirements, are completed on schedule, and are completed within budget?
Savoia: That's a very tough question, and while I am an optimist, I believe that the day where we can regularly produce bug-free code, meeting user requirements, on time, and on budget is a long way off. Especially since we keep building larger and more complex systems all the time and, arguably, software systems are already the most complicated human-made things on the planet.

To give you an idea of what a typical software system would look like if it were physical equipment, imagine replacing software control loops (e.g. for loops) with wheels, conditional statements (e.g. if statements) with mechanical switches, and so on. Even if the size of each physical component was as small as the parts of a mechanical watch, most systems would be the size of houses and full of miniature interlocking gears, switches, levers, etc. There are a lot of moving parts in software, and keeping them all straight is a huge challenge.

While bug-free code may be a long way off, there is still hope for very significant improvements. But we must start to think about testing and quality earlier in the cycle. We must also recognize and face the fact that software testing is a combinatorial problem and that in order to solve it, we need powerful specialized software and lots and lots of CPU power. Those two things, combined, are my mission in life and the reason why I co-founded Agitar Software: To get developers involved in testing and to make them successful by giving them the tools they need.

Alberto Savoia is CTO and senior vice president of R&D at Agitar, which he co-founded in June 2002. Before Agitar, he worked at Google as the engineering executive in charge of the highly successful and profitable ads group. In October 1998, he co-founded and became CTO of Velogic Inc., a pioneer and leading innovator in Internet performance and scalability testing. Velogic was acquired in 2000 by Keynote Systems Inc., where Alberto continued to serve as chief technologist.

Prior to Velogic, Alberto had an impressive 13-year career at Sun Microsystems. His last position at Sun was general manager of SunTest, a highly successful business unit he created and established as the technology and market leader in Java testing solutions. Before SunTest, he served Sun as director of the software technology research group for Sun Microsystems Laboratories, where his staff developed mission-critical and groundbreaking new technology in the areas of software development environments, programming languages and clustering.

Sound Off! -   Be the first to post a message to Sound Off!

Tags: Software unit testing,  Test-driven development (TDD),  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us