The waterfall model of software development, still used by many organizations, has its plusses and minuses. It works very well because it forces the creation of a number of formal documents -- like requirements, functional specification, technical specification and a technical architecture document, if applicable -- before a line of code is written. It does not work well, however, because its projects' time-to-delivery is typically very long. So, quite often you wait for many weeks and months before software delivery occurs and find out too late from the client or the remote software development managers in a distributed team that you have been off course all along!
Agile development takes place in short iterations, in which releases occur in short periods of time. If you have a build, say, every 10 days, you close the communication gap earlier on. The client gets a release and a demo to play around with and can provide feedback on course corrections that need to be made early on in the cycle. You don't need to wait weeks and months to find out that a software delivery was off course all along. The client looks at the software earlier on and provides feedback on whether you are on the right track or not.
Ironically, the fact that agile's short iterations allow requirements changes can be seen as a drawback. Project managers, outsourcing firms, CIOs and others worry that agile would only dramatically increase the out-of-scope features and will result in scope creep.
Another problem I've seen is that pure agile development methods work best when all involved are in the same location. I've certainly seen face-to-face communication resulting in fewer communication gaps than communication via video conferencing, phones and instant messaging. Waterfall's less time-intensive initial stages help make communication less of an issue in distributed development projects.
Hybrid waterfall/agile development's advantages
A hybrid waterfall-agile development methodology is well suited for distributed software development, especially across many time zones. It will make sure that there is a formal Documentation Process first on very broad expectations but there is flexibility in the agile development methodology to make fine course corrections as we go along. Most importantly, we don't wait till the final weeks and days of a project to find out there were gaps in communication all along.
The lack of proper communication between the client and the development team is 90% of the problem in software development efforts. If you fix communication early on, the development goes that much more smoothly. The hybrid waterfall/agile methodology emphasizes communication aspects all through its cycle, as shown in this diagram.
Highlights of the hybrid waterfall/agile development model
The process above is an indicative process. Some parts may already have been performed by the end users' IT group. For example, they may already have a requirements document, technical specification and functional specification ready. Or they may have only a requirements document. The development team selects earlier parts of the process as applicable on a case-by-case basis.
Now look at the key components of a hybrid waterfall/agile process.
- Every project requires a detailed project plan
The project plan is not shown in the process diagram in this tip because sometimes you may need to do it even before you bid for a project. Also, it may sometimes be part of the deliverables somewhere. I use Microsoft Project for this process, but there are other work management tools available.
- Create a question list for clients at each stage of development
Right from the RFP document to a requirements document, at each stage the development team must generate a list of questions for the client to answer, gathering answers about issues which they are not clear or know little. It is better to get these kinds of questions out of the way earlier, rather than making assumptions about anything.
- Some documentation is optional
For example, the functional specification, technical specification and technical architecture documents are all optional and should only be selected and used on a case-by-case basis. If your team is working on a short-duration project to design a part of a web site, creating all these documents may be overkill. The process shows a superset of all documents we may need or the client may need to generate before we start coding anything.
- A high-level design document is always needed
For every project, once past the specification stage, the development team creates a high-level design document; even it is just a couple of pages. This will just show mockups of user interface designs, etc. Do this to make sure that the development team goes over every aspect of the development effort and has a chance to ask questions and clarify things before proceeding to coding.
- Weekly, every-10-day or periodic build release deadlines must be achieved
In the hybrid waterfall/agile model, the project plan breaks up the time available for development and testing into a weekly, every-10-day or -- at a maximum -- every-two-week build-release cycle. On this date, without fail, there is a release.
There may be some flexibility on what goes into a build. Indeed, the development team can have a meeting without clients and decide what goes into a build; but the build release is never delayed for any reason. The reason for this is to build a build-and-release discipline. Every build will be tested formally locally and then and only then released. This also ensures that software is tested many, many more times than in a purely waterfall model and increases the quality of the software indirectly!
- Determine if a requested change requires a change order
While obtaining feedback from clients and creating defects or ticket entries, the first test is to see if the change requires a change order. What is the difference between a simple defect that doesn't require a change order to fix and a flaw that requires a change order? Simple things like font, color or misunderstandings about how something needs to be calculated may all go in as defects. If something involves, say, more than four hours of work, it may be considered for a change order. This is just a rule of thumb and can vary on a project by project basis.
Choosing agile, waterfall or hybrid agile/waterfall
Purely agile and purely waterfall models of software development have worked in specific cases and have failed miserably in others. For distributed software development, whether it is outsourced or in-house, a hybrid waterfall/agile methodology offers the best of both worlds: a semiformal process in the beginning that adds on the nimbleness of an agile methodology. Scope creep is the biggest fear in many software development methods, and this method will limit scope creep by providing a semiformal specification method in the beginning and also allowing for an out-of-scope exception process in the development cycle.
About the author: Nari Kannan is currently the Chief Delivery Officer of V-Soft Consulting, Inc., a Louisville, Kentucky-based software consulting firm. Nari has 20 years of experience in information technology and started out as a senior software engineer at Digital Equipment Corp. He has since served variously as vice president of engineering or CTO of six Silicon Valley startup companies, working in the areas of business process improvement, IT consulting, automotive claims processing, human resources and logistics applications. He can be reached at firstname.lastname@example.org.