Choosing the best software testing tools for your business
A collection of articles that takes you from defining technology needs to purchasing options
Call it a defect, fault, glitch, issue or something else; software testers call them bugs. As soon as the software project becomes too complex and the bugs too numerous to manage on a spreadsheet, teams tend to turn to bug tracking software. Bug trackers help teams with testing by being able to report, manage, track and mark issues as resolved.
Bug tracking tools can often stand alone, and as independent products, they generally provide users a prioritized list of work to do. Bug trackers can also be part of application lifecycle management (ALM) suites or other project-planning tools, from which a prioritized list of new work to do is produced. Here is a list of some common ways to track bugs, starting with bona fide bug tracking software and then including some of their heavier-weight, project-planning cousins.
Before diving into the products, it's important to understand that most bug trackers share a few core features. Bug tracking tools are able to find an issue, give it a descriptive title and create a descriptive narrative of the issue -- what the user did, what he or she expected to see and what actually happened -- so someone else can reproduce the bug. Bug tracking software needs to be able to associate the issue with a given release of the software and assign severity and priority. Bug tracking software should be able to indicate a state -- open, fixed, unable to reproduce or perhaps "closed won't fix." Finally, the user needs to be able to search, sort and report on the bug list, perhaps by severity, priority, version number and so on.
How each tool goes about these tasks is often different, but testers rely on those core areas, regardless of the system they use. Fortunately, these are areas that most bug tracking software manufacturers have agreed upon and can be reliably found in each of the systems.
Stand-alone defect tracking
For companies looking to set up a bug tracking software solution on readily available and inexpensive hardware, have the ability to enter and maintain bugs and updates through a web interface, and have an ability to start small but grow as the organization grows, Bugzilla is an option that allows for both Linux and Windows installations, as well as broad modification ability. Additionally, its availability as a free, open source bug tracking tool makes it attractive for small companies setting up bug tracking for the first time.
Developed and maintained by the Mozilla Corporation, Bugzilla is a web-based bug tracking product that stores bugs in an open source database. To make it work, the organization needs to run the application on a web server (probably Linux and Apache), create a database (probably mySQL), provide its own support and, most likely, simplify the default templates to be easier to read. Setting up and deploying Bugzilla to this level -- so that it is widely useful -- makes it a real consideration for long-term use.
Bugzilla maintains a list of organizations that have public installations (which is a subset of all organizations using it). If you don't want to run Bugzilla yourself, several companies offer to host the data for you, such as devzing, starting at $15 per month.
Although FogBugz started out as a bug tracker, Fog Creek Software realized that it could be used for any new work. Over the years, FogBugz has expanded beyond just bug tracking software to include requirement coverage, requirement scoping and time tracking. It can be used for issue and bug tracking, project planning, time allocation and project management. FogBugz is ideal for companies that want to go beyond just bug tracking and include project planning and forecasting. FogBugz can be hosted in the cloud or run locally.
FogBugz server is designed to run on Windows Server. The client software can run on multiple platforms (Windows, Linux and Mac). For organizations already invested in Windows Server as their standard server platform, FogBugz would be an easy choice for in-house deployment.
FogBugz offers a free edition for two users.
Made by Atlassian Software, JIRA is part of a broader, integrated suite that, like FogBugz, can be used for more than defect tracking. JIRA's great strength is that it has hundreds of add-ons that allow integration with other products, so that activities like test management, continuous integration and configuration management can also take place inside of JIRA. For example, Zephyr, the test case management tool, is available as a JIRA plug-in; and Confluence, the Atlassian wiki for documentation, can work seamlessly with JIRA.
By leveraging existing investments, JIRA aims to work alongside and help integrate with an organization's workflow, rather than replace everything. JIRA is also a great choice for non-technical staff members. Being a programmer is not a requirement to use JIRA, and many of the workflows are customizable without the need for programming skills.
JIRA offers two deployment models and pricing varies based on whether an organization pays for hosted service or deploys in house.
Mantis Bug Tracker (MantisBT) is another open source bug tracking tool with project management capabilities. MantisBT offers a more modern interface than Bugzilla, but like Bugzilla, the customer needs to develop back-end databases, including mySQL, PostgreSQL and Microsoft SQL Server. MantisBT can run on most modern operating systems (e.g., Windows, Mac, Linux and BSD) and requires web server software.
One of its biggest benefits is its ease of installation and use. Setup is very fast and users can be effective in minutes. The system allows for different levels of access, so that projects can be as open or as closely guarded as your organization sees fit. MantisBT would be a welcome product in a heterogeneous development environment, because the server software can run on multiple platforms. Additionally, the server requirements are modest, so even commodity hardware can be used to implement the server, web services and database. Small companies just getting started would be able to get up and running with MantisBT quickly.
As an open source bug tracking application published under the Gnu General Public License, MantisBT is free.
Wider suites and tools that provide defect tracking
The Agile Manager suite by Hewlett Packard Enterprise is fully featured ALM, from requirements to project management to dashboards. For organizations already running Hewlett Packard Enterprise ALM products -- including Quality Center and Unified Functional Testing -- Agile Manager would integrate into these environments as a full quality suite. Agile Manager was built to be run for an internal organization, but has an online, software as a service, web-based element.
Organizations that live in a Microsoft environment using Visual Studio and Team Foundation Server for builds and project management might do well to consider TFS for their bug tracking software. Although the Test Manager portion focuses on testing activities, TFS itself can record bugs as a new work item and allow the development team to make that issue a specific task or target item for their next iteration.
Microsoft offers TFS hosted in the cloud, and also, for security, performance or historical reasons, on a local machine.
Efficiency is the primary object when tracking bugs.
Major defects can hurt your product and credibility. Make sure you test the code, no matter how trivial.
HPE Unified Functional Testing can help to facilitate app testing
FogBugz bug tracking tool reviewed by the experts