Like death and taxes, some problems persistently plague quality assurance (QA) and testing organizations. But new tools, methodologies and standards are emerging to help lighten the burden on QA professionals.
The overarching issues for QA/testing -- heightened by a down economy -- are still schedule and budget, said Judy McKay, author of Managing the Test People. "QA is getting squeezed at the end," she said. "There is more nervousness in the industry about losing jobs, and people are held very accountable to one thing easily assessed, which is date. The problem is, a focus on date is causing a lack of focus on good quality practices."
McKay said not enough resources are being devoted upstream in the application development lifecycle for good quality practices like unit testing. "It's an incorrect assumption that if you skip this it will save time, but it pushes it to system test where it's more expensive. And if you hold to the end date because of budget, then you're cutting out some system testing that really needed to be done. It often pushes fixes into a maintenance release, which is a hidden cost and takes time away from the next project," she said.
Another overarching quality assurance issue, according to Ellen Gottesdiener, principal consultant at Sudbury, Mass.-based EBG Consulting, is the longstanding struggle with scope and managing requirements -- which ultimately impacts QA. "Projects still struggle with appropriately scoping what's in and what's out. Scope should be based on requirements," she said.
McKay said tools that help developers do a better job, like requirements tracking and unit test tools, ultimately help QA, "but a commitment to using them has to be there," she said. "It's amazing how many organizations still track requirements in Word."
But tracking requirements doesn't mean organizations have the right requirements, which is a downside to the tools, said Robin Goldsmith, president of consulting firm Go Pro Management Inc. in Needham, Mass. Requirements tracking tools "have nothing to do with if the requirements are right," he said. "The irony is the requirements community and tool vendors seem not to recognize that."
However, he noted, in the past year or so a number of vendors have been offering products to help with requirements definition. "Most of them are still for capture as opposed to discovery, but they help," Goldsmith said. "They help you with clarification, but they don't get to the question of if the requirements are right or if requirements are overlooked."
Similarly, automated testing tools are helping to address QA issues, but they too have limitations. "How to approach automating tests continues to be a challenge for a lot of QA teams," said Megan Sumrell, a senior consultant at global consulting company Valtech. Some of the responsibility for that falls to the tool vendors, "who are not being honest about the skill set and requirements [needed] to do automation," she said, "They tend not to advertise that to get the best bang you need a highly technical person to run them."
McKay said she's seen a split among organizations, with half pulling back from automated testing because of expense, and the other half putting money into automation to save money and time. For test automation to really help, she said, "you have to be involved early on and working with developers, to make sure they're not changing the UI [user interface] when you're ready to automate."
But the problem with the majority of test automation tools, said Sumrell, is that they're driving automation through the UI. "Everyone sells record/playback, and you can get everyone up and running, but it's not sustainable," she said. "The UI is the most fragile part of your application, so it's going to be the thing that's changing the most. To build an entire automated regression suite through the UI means you're constantly changing the regression suite, so you've got to have a significant number of people constantly maintaining the suite."
However, Sumrell said she has seen QA teams moving toward open source tools that test directly against the business layer or the actual logic -- which "makes testing faster, and scripts don't have to be maintained every time someone changes the UI." One such open source tool is FitNesse, she said.
In addition to better tools, Sumrell said the move toward agile development is helping solve QA problems, because it breaks down barriers between roles and gets testers involved earlier. And, she added, "you don't have to be full-scale agile to get test involved early; it's a big win for QA."
Another potential win for QA will be the establishment of some common terminology, standards and best practices, McKay said, which is something the International Software Testing Qualifications Board (ISTQB) has been working hard on. "The problem is, testers come from varied backgrounds," McKay said. Through the ISTQB, "a level of professionalism is being pushed, [and] we need desperately to be thought of as a true profession."
Oh, and one more thing would help, Sumrell added: "Quit calling it QA." Specialists in testing should be called testers, she said, or put the word "quality" in front of everyone's role -- quality developer, quality architect and so on. "Everybody should own quality."