In the movie Cool Hand Luke, the late Paul Newman uttered these famous words: "What we've got here is a failure to communicate."
In the area of software requirements, agile development methodologies bring to the fore, and try to address, the communications gap between developers and users that has been a longstanding pain point in software projects.
The agile approach to software requirements embraces change, flexibility, face-to-face communication, and the notion of doing enough (but not too much) in a just-in-time manner. Does it help to bridge that gap? Agile practitioners say yes.
"When you're creating software, you're taking knowledge and encoding it into software," said James Shore, consultant and co-author of The Art of Agile Development, and signatory number 10 to the "Agile Manifesto." The problem, Shore said, is that the customer or business expert with the idea has a different expertise than the programmer charged with encoding that knowledge.
In the traditional world, he said, software requirements documents are used to bridge that communications gap. "In the agile world, we believe documents are one of the least effective ways of communicating — you don't see nuances, tone, facial expressions," he said.
So communicating better around requirements requires cross-functional teams that meet together, Shore said. "Agile recognizes that the creation of software is a team activity, and that alone helps bridge the communications gap."
Megan Sumrell, a certified Scrum master and senior consultant at global consulting company Valtech, stresses that flexibility is also key.
"Good agile practices put a framework in place that allows teams to accommodate requirements changes easily, so instead of a lengthy gathering process we focus on just-in-time requirements," she said. "This changes the timing on when you're uncovering requirements and when you're digging into details."
Instead of trying to nail down all the requirements up front, "you're doing it over the course of projects," she added.
Agile also changes the notion of the traditional requirements document, according to Mary Gerush, an analyst at Forrester Research in Cambridge, Mass.
"It goes back to using just enough documentation to convey what's needed," she said. "In some cases there's no documentation — maybe you're writing on a whiteboard and sitting there until the requirements are met."
"User stories are a placeholder for a conversation," Sumrell said. "The team knows it needs conversations to flesh out the details. A use case attempts to define all the details from the get-go."
Some organizations transitioning to agile may do both, she continued. "Some teams are comfortable living in the use case space. If they're doing just-in-time [requirements] and writing use cases around the user story, it can be beneficial."
According to Gerush, it's more about what works than about choosing between user stories and use cases.
"Organizations can use use cases on agile projects, and similarly, they can use user stories on non-agile projects. It really boils down to what works for the organization and the team," she said.
Shore, however, said that it's a misconception (and "pet peeve" of his) that user stories are a requirements tool.
"User stories are not a requirements tool at all; they're a planning tool," he said. "Think of agile as a planning game with strict rules. User stories are playing pieces in the game; they're a way of reminding you of that plan. If you're trying to communicate or capture requirements with story cards, you will be disappointed. The idea is we want to communicate requirements as directly as possible through a face-to-face cross-functional team. It's a point people seem to miss."
That said, Shore also pointed out that it's not true that agile doesn't document requirements. The difference, he said, is that agile methodologies use documentation to record requirements, not for communicating them.
Agile and requirements management tools
Agilists can also make use of traditional requirements management tools, but use them in a "lighter" way. Or they may use open source and commercial requirements tools that are more geared to agile.
"I've seen traditional requirements management tools retrofitted to work on agile projects," Sumrell said, "but most often people are using lightweight open source tools to track requirements and backlogs, because that's what they were built for."
Gerush agreed: "Traditional requirements management tools could be used for agile requirements, but I think you'd need to use them in a different way. The basic capability of those tools still matters in the agile world, but if a company is really embracing agile they might take a lighter-weight approach."
Susan Boers, CEO of Emeryville, Calif.-based Ravenflow, which provides software designed to help with rapid requirements definition, calls requirements "the last frontier." No matter how much discipline organizations have put into the development process, she said, projects still fail because eliciting good quality requirements is difficult.
What the emergence of agile has done, according to Adam Frankl, Ravenflow vice president of marketing, is put a lot of pressure on requirements.
"The people gathering and validating have to react much faster," he said. "They don't have a month to put together a requirements document. What all of agile promotes is getting users involved in the process. Users love being involved, but you can't give them development artifacts they won't understand."