I'm often asked to give a recommendation for the best defect tracking system (DTS). Truth be told, the answer depends on your needs and your resources. Rather than answer that question directly, let's first talk about purpose and value. Once you have a good handle on the DTS requirements of your group, you'll be able to consciously make an informed choice best suited to your organization.
Manage the workflow of your defects
There are several key purposes for using a defect tracking system. First and foremost, a DTS is used to manage the workflow of defects. Most companies manage defects through at least three states: new, resolved and accepted. A new defect has just been reported, often by a QA engineer, but reports can come in from anywhere – support, marketing, even program management or development. The key to the "new" defect state is that it has not been considered—the defect has not received attention from the agile team, or from management, in a more traditional waterfall environment.
When a defect has been resolved, that means a decision has been made about it. The team might decide the defect should be fixed, it could be postponed, it can receive a status of 'won't fix' (or 'won't EVER fix'), or maybe it's even a duplicate of another defect or simply not a bug. When this decision has been made, it tells everyone the defect has been dealt with. It's a trigger for the defect reporter to consider the decision and close or to add more data to change the decision.
Finally, once resolved, a defect needs to be accepted. For a 'fixed' defect, that means it's been retested by a QA engineer and the fix has been approved. For other resolutions, it means QA (or the person originally reporting the defect) accepts the decision. A good DTS needs to recognize the various states in the defect lifecycle, and help you manage defects through each state. Some DTSs end up imposing a monolithic process on top of this workflow, whereas others are much looser in structure. Be sure, before committing to a specific DTS, that your workflow is manageable in the DTS; it's difficult enough to change tools, so adding new process to the new system could spell failure for the initiative.
Manage the status of your project with reporting features
Another reason for a defect tracking system is to manage the status of a given project. Few defect tracking systems stack up well in this area, because to really get a grasp on a project, you need to know the status of other tasks besides defects. However, even with just raw defect statistics (number of defects reported each day, number of defects resolved each day, number of defects by priority/severity, etc.), you can gauge the maturity of your project. Some DTS's include statistical analysis in the product. Others which are built on standard T-SQL databases allow you to develop this kind of report by going directly against the database. (For security and convenience sake, I'd highly recommend securing the database and writing an internal application to report for you).
Documentation and compliance
For many companies in financial, healthcare and other regulated industries, a defect tracking system serves a documentation and compliance purpose. The records in the DTS show effort aimed at complying with various regulations and laws. This use as a historical artifact should not be overlooked.
Build, buy or open source?
One never-ending debate about defect tracking systems is the build, buy or open source debate. I don't think there's one right answer for this! Many companies opt to leverage an open source/freeware solution, and often that works fine for them. Companies with more mature SDLCs or with industry-specific regulatory requirements may find a more mature, commercial application fits the bill. There are a number of commercial-built defect tracking systems to choose from. Some companies, especially more agile/lean organizations, might decide the workflow imposed by open source or commercial DTS's is simply too rigid, and they end up building a flexible, customized DTS.
So what are your options for a DTS today?
- Build your own: I rarely advise companies tackle this project themselves unless they've been running a DTS system for quite some time, and have a strong argument as to why the current DTS, as well as other open source or commercial DTSs, doesn't stack up. It seems like a straightforward project, but to do it right requires significantly more time than one might originally think.
- Open source: two favorite open source DTSs are Bugzilla, and its little brother, Mantis. They're built off the same code base, but have devolved a bit due to developer preference. These are de facto standard selections for organizations that choose to go with an open source DTS. They're built in PHP, and they're extremely extensible. Plus the source is available for modification, if your workflow requirements go beyond their advanced administrative capabilities.
- Commercial solutions abound: HP/Mercury builds a series of quality-related tools in the Quality Center suite. These tools are extremely robust and offer financial, healthcare and other regulated industries numerous features for tracking compliance and adherence. Microsoft Visual Studio Team Edition includes another powerful enterprise-grade DTS. The Visual Studio suite is geared toward team-driven software development, and the DTS component is highly flexible with a variety of templates (including agile workflow templates).
Whatever type of defect tracking system your company chooses be sure the choice is conscious. Start with a list of requirements and desires, and evaluate your options based on those criteria. By planning ahead and thoughtfully reaching a conclusion, your company will maximize the return on its technology investment.
About the author: John Overbaugh, Director of Quality Assurance for Medicity, Inc., is a test leader with 13 years of experience in product and project IT, focusing on quality and defect prevention. John's background covers pretty much everything from consumer applications to high-availability enterprise server applications and highly scalable Web services. John's strengths and key experiences include test strategy, recruiting and hiring (especially for test), outsourcing/offshoring and the test process. His emphasis is effective and efficient software engineering.
John lives near Salt Lake City with his wife, Holly, and his three sons. When he isn't working, John enjoys the outdoors and is an avid photographer. In addition to providing expert answers on searchsoftwarequality.com, John blogs about testing, QA, and engineering at Thoughtsonga.com
John Overbaugh - Home [[email protected]]