News Stay informed about the latest enterprise technology news and product updates.

Software quality assurance more than just testing

Software quality may seem like an elusive goal, but it can be reached by following a comprehensive quality program that spans the software development lifecycle (SDLC), quality expert Linda Westfall told a packed audience at the Better Software conference.

LAS VEGAS -- Software quality requires a holistic approach that includes proper requirements gathering, good programming, thorough testing, sound processes, and skilled people. That is the tall, but reachable, order that Linda Westfall preached during her packed session, "Real Software QA" at the Better Software conference here last week.

As early as the requirements phase you need to be worrying about what quality even is.
Linda Westfall
PresidentThe Westfall Team

This advice may initially appear obvious, but Westfall, president of The Westfall Team, pointed out that quality is often lumped together with testing.

"I'm a firm believer that testing is one of the hardest jobs there is," she said. Testing is a crucial component of software quality, but it is still only one part, said Westfall. "If you try to test quality into a product, you are doomed."

Defining quality
The Institute of Electrical and Electronics Engineers Inc. (IEEE) and the Software Engineering Institute (SEI) both have "sufficient, but not adequate" definitions of software quality, according to Westfall (see sidebar). The IEEE-610 definition stresses adequate technical requirements, and while these are necessary for quality, they are by no means the only ingredient, Westfall reminded the audience. The SEI CMMI 1.2 definition falls short by placing too much emphasis on process. "I've seen really good processes create really bad products," said Westfall.

The focus should be on results and on the value added by the product, Westfall said. Behind the vision of the product are elements such as stakeholder-driven excellence, a process that relies on a feedback loop and the human side of quality. This human element is "integrally important to knowledge work like software," she said. Ability, skills, and the passing of knowledge cannot be underemphasized, Westfall said. There are limits, however. The base of quality should be facts and information, not "gut feelings," she added.

Defects and responsibility
Westfall referred to quality expert Joseph M. Juran on defects. "If you want to build a high-quality product, build one without defects," she told the audience. How does one prevent defects? First, figure out who owns the defects, Westfall explained.

Two definitions of quality
1. A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.

2. A set of activities designed to evaluate the process by which products are developed or manufactured.

SEI CMMI 1.2 definition:
A planned and systematic means for assuring management that the defined standards, practices, procedures, and methods of the process are applied.

"Sometimes it has to be the individual contributor, and sometimes it's management," she said. "If a worker knows what they are supposed to do, then they're responsible for doing it right. If they don't know what to do, it's management's responsibility to get the workers that knowledge."

Too often, companies blame defects on the testing department. "It's got to be the tester's fault, right?" she asked. Usually it's not. Westfall recounted a story of a company she assisted that was in its 37th cycle of development. The problem was that the company was still letting requirements be changed between cycles, she explained. However, this information never made it back to the right people.

"They didn't have a feedback loop telling them what was wrong," said Westfall. Without that loop, the responsible parties unknowingly made the same mistakes 37 times in a row.

Requirements and stakeholder-driven excellence
One of the most common mistakes Westfall sees is attempting to gather requirements without identifying all of the stakeholders first. In order to achieve stakeholder-driven excellence, "you have to know who these people even are if they're going to drive your processes," she said.

Stakeholders include a larger segment of people than one might initially suppose, according to Westfall. There are direct users and indirect users -- a secretary might use software and pass along the information to a boss, making the boss an indirect user. In addition, Westfall includes suppliers, their developers, and distributors among the stakeholders. Other less obvious stakeholders may be regulators and, perhaps, "society at large," she added.

"As early as the requirements phase you need to be worrying about what quality even is," said Westfall. Software creators should engineer requirements to achieve three types of stakeholder quality: basic, expected, and exciting.

Expected quality requirements are those expressed by the stakeholder. Basic requirements are not necessarily going to be expressed.

"In software, the customer is never going to tell you about these," she said of basic requirements. "This is where you need the knowledge and skill."

Westfall used the example of a windshield on a car; a stakeholder would not directly ask for a windshield on a car, but a car maker would be foolish not to include one.

More information on software quality practices
Quality process first; quality testing second

How to build better software using lean manufacturing, continuous quality improvement practices

How poor management skills jeopardize software quality

"Exciting" quality requirements are, as their name suggests, unexpected and helpful elements. A cup holder in a car was once an exciting feature, said Westfall, and it is a good example of how requirements evolve. A cup holder became an expected feature and is now a basic one.

Effective requirements engineering is extremely important to software quality, Westfall said. She estimates that "45 to 60% of defects in the field are due to bad requirements specification."

Process improvement
Following a good process is integral to quality, but is still only one aspect of quality, Westfall stressed throughout her session.

"Don't improve your process to the detriment of the entire system," she said.

Westfall did not champion one methodology over another, but outlined some practices that a good process should have in place.

To demonstrate, Westfall displayed a chart of process improvement that linked processes, products, defects, and lessons learned in a circuit with one another. In this circuit, products and defects are connected through defect detection -- a key element of quality. Defect detection includes verification and validation (V&V) and rework. "You can greatly reduce the number of defects through prevention activities" such as these, she said.

The cornerstone of a good system, said Westfall, is that it prevents people from making the same mistakes. "Mistakes result in failures," she said -- and failures are a symptom of a larger problem.

Westfall advocates root cause analysis to eliminate mistakes, and she defined this analysis as "the process of assessing sets of product defects or process problems to identify their systemic cause(s)." For more ideas on process improvement, Westfall recommended SEI's IDEAL (SM) model.

People are your number one investment
"The biggest contributing factor to quality is the capabilities of your people," said Westfall. She discussed several improvements companies can make in order to get the most out of their people.

Workers must be given the time necessary to complete their tasks. "If you compress too much, you extend," said Westfall. She praised the Agile timebox approach. "In Agile you keep the schedule static and change scope within the schedule," Westfall said.

Environment is another aspect that can greatly affect the productivity of workers, added Westfall. Interruptions and noise levels should be controlled. Access to conference rooms is a particularly important issue in software development. "People shouldn't have to go on a safari for two hours just to find a conference room," she said.

Additional barriers to communication include irritating motivational posters that cannot be removed from walls and a scarcity of chairs. Westfall referred to the "furniture police" made famous by the book, Peopleware: Productive Projects and Teams. Considering the large amount of time developers spend working with one another, a lack of chairs can be a real nuisance and time waster for developers, she said.

Keeping quality visible
Ultimately, information on quality needs to be visible to all of those involved. Audits, reviews, and metrics are ways to spread this information around. "Audits provide independence and objectivity," said Westfall. They may be particularly useful in organizations where people are afraid to speak up, she added.

Peer reviews should have "clearly defined entry and exit criteria," explained Westfall. Examples of peer reviews are walkthroughs, desk checks, and inspections. "Post project/activity reviews and retrospectives" serve to bring people into the lifecycle and eliminate process mistakes.

Among its benefits, software quality "lowers maintenance costs and creates happy customers and happy developers," said Westfall.

Dig Deeper on Topics Archive

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

Let's turn it around: Testing may help Quality Assurance but it's not QA at all. Taking temperature doesn't treat the illness.
(Good) Testing produces actionable feedback. But someone has to take it and do something.
Testing also doesn't prevent defects - it discovers them. But the discovery may help to prevent defects from slipping into production.

In all, I've got a mixed feeling about the article. Some points are right on - "People are your number one investment".
Some are questionable - definition of quality by requirements and absence of defects.