I have seen many articles touting the latest and greatest hip methodologies, trying to find the Holy Grail for creating commercial software: inexpensive, high quality and delivered on time. Remember the old adage: high quality, low cost, and delivered quickly -- pick two? Often the goal of these agile methodologies is to have as little documentation as possible. I am all about timely communication and rapid time to market, but you need documentation. What concerns me is seeing everyone run away from the dumped "waterfall" bathtub without checking to see if the baby has to go, too.
The only music that upper level management usually hears when agile development is talked about is: "You mean we can get it cheaper and faster this way?" They get positively giddy at the thought of saving money and time. Wouldn't you? As a former software development manager and senior program manager, I am not fond of seeing software developers sitting around waiting for the functional specification to solidify into a fossilized museum exhibit before anyone can start coding. But in the long run, foregoing all documentation isn't going to improve anything.
Delivering wrong functionality -- Recipe for disaster
Yes, the goal of all of these new approaches is to ensure that the software development process is more nimble to deftly handle changing customer needs. Rigid signoff software cycles have to make way for more agile forms of methodologies that allow for that feedback loop. But the question still remains: Why is everyone trying to kill the software functional specification?
Software developers appear to be the ones driving the bandwagon, since they are often tasked with writing the functional specification. You can lose a lot of money when your expensive code cutters are just writing documents. But this fear often translates into calls to use as few written words as possible to communicate functionality.
After all, the developers think, they understand how the products works, why can't everyone else? The reasoning is they believe you will ask them to explain it if you don't understand. But then you are stuck trying to absorb the information from the developer's point of view, maybe writing down a few key pieces of information before being sent on your way for having overstayed your welcome.
The question is who the "You" is here. "You" is really those pesky customers and the rest of the company that directly supports a product. I hope I am not revealing a closely guarded secret when I say developers are not the only people who provide important products to the release. The other components being delivered at the same time, whether they are tangible or implied, can be at least the following:
- Software that meets the company's executive revenue initiatives
- Software that clearly meets the customer's needs
- Comprehensive test plan coverage for the software that ensures quality
- Customer documentation that is clear and easy to understand to support the software
- Documentation that successfully supports field support
- Help desk documents that allow support technicians to easily support a release
- Clear and effective training classes that support the software
- Marketing campaigns to effectively target the right customers that will benefit from the product
- Sales campaigns that effectively sell to those customers who should buy the product
Bringing up baby
Let's look quickly at how typical software for a new product gets born. A group of people sit around with a customer and come up with what looks like some good ideas. Maybe one member of the group is the company's star developer, Charlene. She says, "Yeah, let me see if I can whip something up." She comes back in a few days with a prototype full of hard coded references, but the thing works just enough to make an impression on the customer. The customer loves it, and the company gets the job.
The key to understanding what happened as Charlene works her magic is she really started out with something like "Hello world!" -- especially if she is using some new language and/or technology. Then, as she feels comfortable with what she wrote, she adds more to it. But through the process, as the software grows in complexity, she retains an integrated knowledge of what is in there. She has time to practice her knowledge.
You then add some smart engineers to write the real code. The software is still small and easy to grasp at this stage. The whole department of developers starts adding more string until the ball grows larger and larger and more complex. Three-fourths of the way through the project, a new engineer joins the group. He has to learn the code by diving into it, reading some implementation specs and making a pest of himself with the other engineers. Most engineers will tell you that in this situation, with a very complex product, they don't feel comfortable contributing for quite some time. Six months to a year is not unheard of.
Throughout this process, the developers on the project are again able to layer their understanding of what is going on in the software as it builds in complexity from the tiny "Hello world" germ. No one else on the development team is in this position. Without a clear road map, the software is an opaque mass with little on the outside to tell what it does on the inside.
Bring in the rest of the groups
What about marketing, training, sales, quality assurance, tech support and field support? They may not have the time or be able to read the code to figure out what the software does. In fact, you do not want them doing that. What if the code was written incorrectly in the first place? What if there is a feature missing? Do we now ask developers to become accountable as marketers?
Now imagine engineering has delivered finished code as production ready, or at least ready for testing, to the rest of the other individuals on the cross-functional team. Upper level management is expecting marketing, sales, tech support, training, and QA to understand the product well enough so that they can provide their products to support the software. If a new engineer takes a good six months to get integrated into the process, how long will it take these folks to understand it well enough to do their jobs? This is what we are expecting of anyone outside of engineering if they are not provided clear functional detail on how the product works.
Unraveling the ball of string for everyone else
OK, let's have each one of the other members of the program team sit down with a developer to figure out how the product works based on their frame of reference. When do you start? Do you start at the beginning? Do you have weekly meetings with the developer to understand what he is writing so that you understand what is going on? Maybe the code is not at a place where that is possible. Do you wait until the code is ready for testing to find out how it works? By then, however, its complexity level has reached its maximum.
Quality assurance not only wants to know how it functions, but also what happens when it doesn't. Will the engineer, who has all of this in his head, remember all of the complexities and all the error paths? Has he even coded them? Do the other team participants have the same six months that the new software engineer was given to understand the product before they are productive?
Because sales and marketing are often required to release their parts at the same time as, or even earlier, than the software, they need a detailed understanding of how it works before the code is even done. Training needs to provide accurate usage information for the customer. How many products are so complicated out there and have such poor documentation that most customers never know how to use all of the features?
Pay me now; pay me much more later
Do you really want every member of every other group spending time with your engineers, trying to unravel enough information so that they feel that they have a comprehensive understanding to do their jobs? Even if you have cross-functional Scrum meeting throughout the design and development process, the level of detail for all the stakeholders is rarely there. Yes, there are engineering design meetings, too, but they generally focus more on implementation with specifics of functionality scattered throughout the discussion or often held on an ad hoc basis.
When development is done this way, you see that elimination of those "tedious" documents shortens the development time up front. But it gets added at the end big time with a nightmare of frustration, misunderstanding and lack of accurate information for the rest of the supporting organizations.
Yes, your engineers are an expensive resource. You want to leverage them appropriately and efficiently. But if the rest of your other groups are not able to produce a quality product to support the software, you do not have a useful quality product.
You need to start the documentation early enough and align your organizational players correctly to ensure that the end result is that you increase, not decrease, your development throughput. Instead of forgoing writing the requirement specification, consider having the document written by someone not in engineering. For the most effective resource management start the design effort well before the completion of your present software project. This will allow your engineering department to continue coding its present release.
Consider having a program manager, product manager or even QA engineer trained to take on the task as the design leader and scribe. Yes, the document should be a collaborative effort that reflects engineering design input and their needs to be able to architect the solution. But having someone other than engineering create the document ensures production time is not blocked and the document produced is useful and understandable by all participants, not just engineering.
If the document is understandable enough, then all the "customers" can participate in the effort. Changes in direction will result in words on the paper being erased, rather than expensive code abandoned. Maintaining the document during the development process can be accomplished
About the author: Nancy Knettell has over 15 years of experience in senior program and software development management, as well as hands-on software and quality assurance engineering experience for companies such as United Technologies and Philips Medical. She currently works as a software quality process consultant.