|Robin F. Goldsmith, JD|
The analysts' unfamiliarity with how to estimate their work's effort and duration highlighted the divide between traditional project manager (PM) and business analyst (BA) role definitions. While usually considered a project management task, estimating obviously also matters to BAs.
This Part 1 of a two-part article describes top-down estimation techniques and their applicability to estimating requirements definition effort and duration. Part 2 describes bottom-up estimation and suggests an initial set of tasks that probably would be needed to define requirements and thereby estimate time and effort.
Requirements and estimation relevance for agile
Since classic agile eXtreme Programming (XP) project teams consist only of a user representative and two or more programmers, and since some would say that it takes negligible time for the user representative to provide an XP project's requirements in the form of a brief user story, one could question whether BAs, estimating and requirements definition are relevant to agile as well as to more traditional projects.
Regardless of methodology, adequately defining requirements is an important part of any project and depends on accurately estimating and allocating suitably skilled resources' time. Even trained BA specialists often fail to define requirements correctly. Effectively gathering and analyzing requirements data, especially from many sources, is difficult and takes skills beyond those one may have merely by virtue of being a user representative. Increasingly, many agile projects do include BAs.
This article applies to agile as well as other approaches. Contrary to impressions given by some agile advocates, the alternative to agile is not necessarily lengthy inflexible phases endeavoring to define all requirements in detail before proceeding to do all design before doing any coding.
Informed planning, in fact, is the most truly agile technique and depends on appropriate requirements definition, which involves more than a single user story. It is iterative but starts with discovering high-level REAL business requirements that provide an informed context for identifying content to elaborate subsequently, such as the suitable set of user stories for agile development.
Several of the chatting BAs seemed to be searching for an estimation template as the solution to their dilemma. Templates, checklists and similar tools are helpful step-saving reminders for people who already know what they are doing; but such devices are not at all sufficient by themselves for teaching what to do and how to do it.
For example, airline pilots and surgeons both use various checklists and templates to make sure they've done everything necessary for flying and surgery; but I wouldn't want a novice guided only by the checklists piloting my plane or operating on me. The Internal Revenue Service tax forms are templates. Do I need to warn further about templates?
General estimating principles
All estimates are based upon translating information about some known reference to the target – the thing being estimated. The accuracy and reliability of such estimates depends on the extent to which:
- The reference is similar to the target.
- The size of the reference is known and can be scaled to the size of the target.
- The reference's actual work effort/cost and duration are known.
There are two major approaches to estimating. Top-down estimates make a single estimate for the whole thing and allocate that estimate among the major work pieces. For instance, contractors often estimate the total cost of building a house and then apportion certain percentages of the total to the roof, plumbing, framing and other components.
Top-down estimates are quick and frequently are the favored approach of top management. Unfortunately, one's position at the top of an organization often has little to do with ability to make accurate top-down estimates. Top-down estimates are very likely to be wrong, usually by a large amount, because one, two, or three of the above reference criteria have not been met.
For instance, the most common way that testing time and effort is estimated is top-down as a ratio of estimated development time and effort. It's questionable whether development is a suitably similar reference for estimating independent testing, especially considering that such rule-of-thumb ratio testing estimates seldom are adjusted to reflect variations in needed testing work due to factors such as whether or not reviews and inspections were done, amount and caliber of testing performed by developers, and inclusion of performance, security, and other very demanding additional special tests.
Development and testing often don't scale with each other. A one-line code change may necessitate total regression testing, repeating all of the tests for the system. Moreover, most people's experience and studies such as The Standish Group's CHAOS Reports, tell us that estimates of the reference (development) usually are low by around 200 percent.
Parametric estimates are a sometimes more reliable form of top-down estimating. The reference consists of a set of quantified parameters and formulas for relating the parameters. Thus, most contractors actually base their house-building estimates on a handful of parameters for which there is extensive documented historical data, e.g., square footage, house style, materials, and quality/luxury.
Automated tools for estimating system development projects are parametric. They rely overwhelmingly on one dominant parameter: expected size of the program or system. Size ordinarily is measured based on the program/system design in either KLOCs (K for thousand, LOC for lines of code) or function points (FPs). The number of lines of code needed to perform a function varies considerably depending on programming language, which makes cross-language comparisons inappropriate. FPs are an often-preferable, language-independent, artificial measure of program or system size, based upon counting several types of functions according to an extensive set of standardized counting rules.
Unfortunately, these fairly-well-established design-based development estimation methods don't apply to estimating the requirements effort. Program size is estimated based on design, which comes after the requirements effort. Agile may not have design documents; but programming nonetheless implements a design intended to satisfy the requirements. It's not just agile development where designs are only in the programmer's head.
Requirements effort may be related to the size of the requirements; but I know of no reliable way currently to measure requirements size. Moreover, since requirements are the starting point, there's no apparent prior reference measure which would be reasonable for estimating the size of the requirements to be defined in the present project.
Part 2 of this article will describe bottom-up estimation and suggest an initial set of tasks that probably would be needed to define requirements and thereby estimate time and effort.
Robin F. Goldsmith, JD is President of Needham, MA consultancy Go Pro Management, Inc. He works directly with and trains business and systems professionals in requirements, quality and testing, metrics, ROI, and project and process management. He is the author of Discovering REAL Business Requirements for Software Project Success.