Gathering and managing software project requirements
A comprehensive collection of articles, videos and more, hand-picked by our editors
The terms gathering requirements or eliciting requirements are commonly used to refer to the process of finding out what the mandates are for a given project or piece of software. But neither term accurately describes the actual process and, increasingly, may not even match participants' perceptions, especially for Agile projects.
Gathering and eliciting easily are confused with implying a separate lifecycle requirements phase. Because Agile projects do not characterize their work as phases, some may feel Agile projects do not gather or elicit requirements. However, gathering and eliciting are activities that do occur in any project but may not be explicit in Agile. Thus, a product owner may state requirements without gathering or eliciting being evident, possibly because there wasn't any. Similarly, Agile developers may characterize their requirements activity as conversations rather than gathering or eliciting.
Regardless, there is a more fundamental issue that makes the terms inappropriate. Gathering implies requirements are sitting around fully formed when, in fact, most requirements are hidden, though unintentionally. Eliciting more accurately describes the process of revealing not-so-observable requirements. However, both terms are misleading because:
- You actually elicit or gather data about the requirements rather than the requirements themselves.
- Requirements are discovered by analyzing the data.
For these reasons, I refer to the process as "discovering requirements," which consists of integrated interactive data elicitation and analysis, which, in turn, may be followed by additional analysis.
I believe discovery's purpose is to understand the project's real challenge, the value of addressing it adequately, its causes and, what I call, the real business requirement deliverables that will provide value by achieving objectives. Only then is it productive to consider requirements of software and ways to meet business needs by satisfying the business requirements.
A business analyst or other discoverer -- such as a product owner, developer or project manager -- finds data from some source. Techniques vary depending upon the type of source. You can use the following elicitation techniques to help discover business requirements and software requirements. I'll point out practices that differentiate more-effective from less-effective requirements engineering. Less-effective discoverers try to define software requirements without first determining the business mandates they must meet, which will likely lead to creep and wasting time building the wrong thing.
The typical image of a business analyst involves someone conducting interviews. Interviewing is the fundamental and most common data gathering technique. Interviews usually are one-to-one; that is, one interviewer questioning one interviewee. I've found it helpful to have two interviewers so each can pick up on things the other may miss, but few organizations take this course because of its added cost.
Try following these helpful hints of more-effective interviews:
- Prepare for interviews. Learn about the subject area and determine what types of information the interviewee is likely to know about, which will be the basis for determining which questions to ask.
- Don't interview multiple people together. This is ineffective for so many reasons -- starting with the "you're not important" message it sends to interviewees. No one receives adequate attention and the presence of others often limits what people will say.
- Ask open-ended questions. Such questions are best for initiating topics because they allow the interviewee to set the direction, telling you what the interviewee thinks is important. Don't initiate topics with close-ended questions that invite short answers and shut down further discussion.
- Ask follow-up questions. No matter how well you prepare, you're there to find out about things you can't possibly already know. Probing draws out the otherwise hidden data that initial responses to open-ended questions may only hint at and that ineffective interviewers (who are all too common) often are oblivious to. Welcome unexpected answers, for they will lead to questions you hadn't planned on asking. Those questions can provide some of the best data.
- Listen and analyze what you hear. Think about implications of what the interviewee says, probing further and asking for clarifications to better understand what and why it was said. Offer feedback and periodic summaries so the interviewee knows you are listening and understand what was said.
Two additional techniques are powerful, somewhat indirect variants on interviewing. Because they require greater analysis skill and more time, these methods are used far less than they could or should be.
- A few authorities emphasize observing operations, which often is called job shadowing. You can learn a lot by watching experienced workers do the jobs that requirements must address. Their experience makes them likely to skip over telling you about important aspects of the work they take for granted, but that can become more evident when observed.
- Even more powerful is learning to do the work, because it can make you aware of things that are not evident by just observing a skilled worker. This approach also sends valuable signals that build rapport and reliability in ways that observing often does not achieve. Observing operations and learning to do the work are supplements, not substitutes, for interviewing and other fundamental elicitation techniques.
Group data gathering
Several one-to-many group methods invite responses and interaction from all group members, typically when they are gathered together at the same time. These elicitation techniques provide ways to overcome the weaknesses of interviewing multiple people together.
Brainstorming is the best-known group technique. Brainstorming takes place in two separate sequential activities, roughly corresponding to the elicitation process and analysis. The first part of brainstorming is intended to raise as many ideas as possible. Therefore, there is little discussion or censoring of raised ideas other than as a means to open awareness to additional ideas. The second part of brainstorming analyzes and organizes the ideas, often through affinity grouping of ideas written on individual sticky notes that can easily be rearranged into meaningful groups. Ideas that surface during analysis are added to the clusters.
Focus groups bring together knowledgeable, interested parties to examine a relatively narrow topic in depth. Because focus groups can be difficult to form and are relatively expensive, they tend not to be used often for eliciting project requirements, but are mostly used to shape marketing campaigns where revenue payoffs can be enormous.
In contrast, requirements workshops -- or Joint Application Development/Design -- bring together as broad a group of stakeholders as possible, but generally must stay relatively high-level. They cannot fully address a topic of interest to one stakeholder without losing the rest of the group's interest. Most are led by a trained facilitator focused on keeping everyone involved.
Requirements workshops are used widely to elicit requirements, in part because several prominent authorities tout them as the main -- or even only -- elicitation technique to use.
Requirements workshops cut through communication difficulties by bringing everyone together and creating a sense of ownership. Although good for kicking off a project, they need to be supplemented with other elicitation techniques better suited to producing relevant detail. Also, I find that even though they are called "requirements" workshops, they easily can skip discovering real business requirements and turn into design of presumed products that many people mistakenly think are the requirements. Too often such premature focusing on products leads to products that fail to satisfy the (inadequately identified) real business requirements.
Surveys and questionnaires are different types of group elicitation techniques because the discoverer and data source are not interacting directly. The big advantage of these methods is that they enable collecting data quite economically from a large number of people. Such benefit often is illusory, though, because survey and questionnaire data tends to be highly unreliable and may necessitate considerable additional follow-up to truly understand it. Frequently, a survey's main finding is that other questions should have been asked instead. Surveys are most useful for capturing clear factual information. Less effective discoverers tend to rely too much on surveys and questionnaires.
Technically oriented elicitation techniques
Many people swear by prototyping for elicitation. It's a valuable technique, but not in the way most advocates believe.
Prototyping involves quickly creating a working version of the product so users can try it out and react to a more tangible representation than a requirements document could provide. To enable quick creation, prototypes mainly focus on the graphical user interface (GUI) and shortcut the internal processing logic "guts." The problem is that most of the business requirements and value are actually in the guts, and, thus, easy to gloss over with a prototype. Prototypes generally shift the focus from the users and their business needs to the product the developer intends to build.
Increasingly, tests are being recognized as a way to elicit requirements. Static testing reviews requirements directly and often elicits needed additions or changes. Most reviews deal primarily with form issues, such as clarity and testability. I use more than 21 ways to review requirements that also identify wrong and overlooked requirements, which generally are the more important content concerns that typical form-oriented reviews usually miss.
Agile development, specifically Extreme Programming, has popularized having developers write executable dynamic tests before writing the code that then must pass the tests. Such executable unit tests and user story acceptance tests often are considered forms of requirements. Even if thought of only as tests, their concrete nature helps identify and communicate misunderstandings that can elicit new or different requirements. However, note that using such a method in no way ensures how well it is used, and developers often create poor tests. I find that more-effective requirements and tests result from more-informed test design before coding, whether or not all the designed tests are constructed and executed.
Several elicitation techniques can be used by a business analyst without tying up other people's time. However, these are often overlooked by analysts and, surprisingly, are unfunded by managers who don't understand the techniques' value relative to their costs.
Reviewing project-related documents is an obvious and frequently used form of elicitation. Such documents include product and system documentation, problem reports and change requests. Reviewing documentation of related systems enables interface analysis, which is an important element of requirements that otherwise can be overlooked easily. Similarly, a discoverer can use data from documents to define and analyze processes, which can reveal requirements related to overcoming delays, errors and other forms of waste.
Effective requirements discoverers are more likely to invest their time in background research that is not specific to the project. Books, articles, classes, product descriptions and reviews, webinars, and websites can tie up a lot of time providing a basis for better understanding the business situation, but they do not identify the particular project's requirements.
Effective requirements discoverers use more than one of these various techniques. They examine and follow up on findings and invest energy in becoming better at requirements discovery techniques.
Importance of sequence during requirements elicitation
Ask the expert: How do you structure requirement specs?
Agile doesn't prevent effective requirements documentation