When I started working as a tester in the mid-1990s, I learned that the basic transaction between developers and testers was that developers sent code to the testers, and testers sent bug reports back to the developers. The phrase "over the wall" was a popular way to describe this transaction.
But I made my bug reports with a twist. By a pure accident of history, I had learned to use the debugger for my platform in my very first software job. So when I found a bug, I found it a simple matter to fire up the debugger in order to include in my bug report something like "On line 1565 the program has a hard-coded error message 'foo' but the operating system is reporting error 'bar'".
Some developers appreciated this. Other developers were nervous that a tester was reading their code. Regardless, reporting bugs in this manner saved a lot of investigation time.
After a while, I figured out that if I removed the step where I wrote the bug report, I could be even more efficient. Sometimes I'd visit with the developer to demonstrate the issue and get it fixed. Other times I would send an email saying "hey, could you make this code change for me so the software won't misbehave?" A few times I even wrote some production code myself to fix a little bug, just because it was easier for me to do it than to bother developers that were doing more important work.
The only drawback to this approach was that defects reported to the institutional bug-tracking system became pretty sparse. After a while, the defect tracking system only contained bugs that were either very complex, or that would likely never be worked on at all. According to the standard approach at the time, this was a bad thing because management was unable to track who was doing what to the code when all the defects were just being fixed on the spot between the testers and the developers. On the other hand, the software itself continued to get better and better, and the system releases came more quickly with fewer problems than ever before, so we were a pretty happy team, even though we were violating many of the best practices of the day.
I was doing this work at about the time that the Agile Manifesto came out. It was great to have such an important document to justify the approach we had evolved all on our own. I coined a term at the time about bugs that at one point had some currency among agile practitioners: Just Fix It. Meaning: if you have a bug, and you Just Fix It instead of entering it into the bug-tracking system where it has to be marched through the process and ultimately retired, you can save an immense amount of time and effort. People over process.
The communication problem
I continued to work in a more or less agile fashion through a number of jobs, trying my best to get bugs fixed as early as possible. I used some unusual approaches to keep from forgetting certain bugs that couldn't be fixed right away, such as index cards on a wall, whiteboards and wikis. They were all pretty successful, and didn't require any of the multi-state overhead of manipulating defects in traditional defect-tracking tools.
But ultimately I changed my mind about the value of bug reports, at least under certain conditions.
Bug reports in context
There are two situations where bug reports are useful.
In 2006, I started telecommuting. When working remotely, the communication bandwidth among team members is drastically reduced in comparison to the communication bandwidth shared by a common-located team. On a distributed team, a lot of necessary communication shifts to the written word. On a healthy distributed team, people share voice and computer screens a lot, but the lack of ambient information in the form of overheard conversation, posters on the wall, posture and attitude is all unavailable.
So other tools have to substitute for such ambient communication. There are a number of tools I really like for distributed teams: IRC; wikis; VNC; messaging. And, it turns out, that bug reports are an excellent idea for a distributed team.
And while I have not experienced this myself, I suspect that there are people on very large teams facing some of the same communication issues faced by distributed teams. The same communication channels that serve distributed teams would also be useful for very large teams.
The how and why of bug reports
At the time I started testing software, defect-tracking systems were large, expensive, inflexible, and unwieldy. Today defect-tracking systems like Bugzilla and Jira are highly configurable. This can be good and bad.
I have found that it is worthwhile to spend significant effort configuring one's defect-tracking tool to satisfy as much as possible everyone with a stake in tracking defects. Developers might want a particular set of information in a template; testers might want certain notifications about the state of the bug reports; managers might want certain reporting capability. Remember that the ideal situation with regard to a defect is to Just Fix It, and regardless of how one ultimately configures the defect-tracking system, the optimal solution will likely be the one that best approximates the Just Fix It idea.
Testers provide the input to any defect-tracking system, so it is critically important that defect reports be of the highest quality. Here are some guidelines for creating good defect reports:
- Explain succinctly why the behavior is a defect
- Explain exactly how to reproduce the defect. This is usually step-by-step instructions
- Do not editorialize; report simply facts, not opinions
- Supply relevant information about the test environment like versions, operating systems and such
- Do not supply information irrelevant to the immediate issue at hand
A good bug report is a finely-crafted artifact. If you are going to be creating such things, make them as good as you can possibly make them, and the entire process of handling such reports will become a smooth part of the regular operation of the team.
This was first published in December 2009