Software testers facing six big challenges today, StarWest keynoter says |
 |
By Jan Stafford
09 Oct 2009 | SearchSoftwareQuality |
|
|
Stop describing standard software test processes as best practices, because tomorrow a better practice could be discovered, said testing veteran Lloyd Roden during his StarWest 2009 keynote in Anaheim, Calif.
Roden, a consultant for U.K.-based Grove Consultants, list of today's biggest testing challenges included a few surprises, such as the risks of best practices and test managers not doing testing. Other challenges he named include getting test estimation and metrics right, dealing with reduced resources.
Here's a rundown of Roden's challenges list, along with his views on each topic.
- Don't require your expert testers to adhere strictly to best practices.
The Dreyfus Model for skills acquisition showed that people who have gained expertise via experiece perceived things differently than novices, because they analyzed situations based on their base of knowledge. The expert's ability to analyze situations makes it possible to tailor practices to fit a particular situation.
In test organizations, said Roden, novices and advanced beginners need best practices to "protect them from themselves" and avoid serious mistakes, Roden said. As skill levels increase, testers know the practices and use them for reference but also add their own tried-and-true methods. Experts, however, should be subverting best practices by discovering new and better ways to do things.
"An airplane pilot doesn't refer to a manual," Roden said. Likewise, the experts on a test team should not be required to go by the book. "Best practices can stifle creativity and frustrate your best people," he said, advising managers to encourage creativity.
- Add quality and monitoring to test estimations
"Test estimation is often wrongly performed, because it leaves quality assurance out of the equation," Roden said. Test estimation is typically based on the equation of effort + people = schedule; but it should be effort + people + quality = schedule.
"Once we've estimated quality, we must monitor quality as well," Roden said. He advises use estimation iterations utilities to analyze the equation: - bugs available to find in next iteration = bugs - bugs fixed + new bugs + nested bugs.
- Don't lie with metrics.
"Be careful with figures. They must be accurate to carry any weight at all," Roden advised.
Most testers and test managers may not create incorrect test metrics intentionally, but the way test results are measured may not represent those tests' true value. Sometimes, a test metrics presentation is designed to get upper management's approval, said Roden. More often, they don't know the best way to present results.
Roden recommends using basic statistics and meaningful analysis. Also, he said, avoid measuring success by counting test cases. "We need to understand what's in the case to be able to measure it's worth," he said. "Report to management what's in the test cases rather than the numbers. That's my challenge to you."
- Using the resources at hand
"I've been in the software industry for 28 years, and I've never know the economy to be as it is now. It's global," Roden said. Gesturing toward the audience, he added: "For everyone here, there were probably 10 people who wanted to come but they had no budget for it."
To do well in staff- and budget-crunched time, analyze your team and tools carefully. "See what strengths you have and what strengths your tools have and build on those strengths," Roden said, suggesting the use of psychometric tests.
Even though it's a buyers' market today, don't mislead clients and vendors to think you can do more than you can. Most successful relationships are built on trust, not speedy delivery or pricing. "Be open about what budgets you have for tools, training and consultancy," Roden cautioned. "Honesty is imperative so that trust can be built on both sides."
Don't underestimate the impact of inadequate staffing and budget on morale and productivity. Roden advises not pushing project too hard and putting some slack into schedules. "You don't want your people to suffer burnout," he said. "You've got to have some slack."
Use free or cheap tools. Some organizations have lockdown on tools, he admitted, but he's seen budget issues open room for negotiation.
- Ax complexity
"At every opportunity, he said, ask: 'Do we need that complexity?" He referred to a 2002 Standish Group study, which showed that 64% or software products' functions are not used.
Too often, people see simplicity as uninteresting and not sellable. Roden recalled advising Nokia to create a simple cell phone for the ever-growing senior citizen marketplace. "They laughed," he said. "Then Sony put out a phone like that the next year" which was a big success.
- Test managers don't truly know what's happening in testing.
Too many test managers don't do testing themselves anymore, Roden said. This leads to poor test estimation, low tester morale and missing mistakes.
In his consultancy work, Roden asks test managers if they do any testing. He hears a lot of excuses, like they don't have time and are busy managing testers. One of the worst excuses he's heard is, "I can't perform testing, as I don't know the application." That flabbergasted Roden, who asked the audience, rhetorically: "How can you estimate if you don't know the application you team is testing?"
In Roden's opinion, test managers must test, so they can assist when more testing is required and improve estimation skills by knowing how long tests take. "This is non-negotiable," he said. "I'll guarantee you'll gain credibility in your organization if you get involved in testing."
Tackling a challenge
Be careful when identifying challenges and choosing which ones to tackle and in which order, Roden advised. Make a list, or look at his list and identify a challenge that affects your team. Then make an action plan which includes a series of steps to take. "Choose your battles well, and then don't give up!"
|
|
|
|
