The test project is the technical effort of estimating work, planning the test scope and strategy, effectively managing test execution, and reporting on status and risk. Every project involves tradeoffs between features, time, quality and costs. It's the test manager's responsibility to manage the details of the test project by possibly providing information to stakeholders in terms of project estimates for the amount of work required, actively managing the test effort, and providing information about product quality. This tip focuses on the documents that might help when managing the testing project.
In our webcast on "How to plan your software test project" we outline some common test planning artifacts that one might find useful. These include the following:
- Test strategy document
- Test plan document
- Test estimates
- Test project plan
- Other documents based on your context
Test strategy document
We have used the test strategy document to define the strategic plan for what aspects of the system we plan to test, our approach to testing, and to capture the details around constraints, assumptions and risks for our testing.
The test strategy document might capture the following information:
- Scope of the testing
- Quality criteria
- Feature context
- Technical architecture
- Operating environment
- Interaction with other features and systems
- Other content items as appropriate to your project
- Testing dependencies, assumptions and constraints
- Testing objectives
- Testing approach
- Types of testing
- Testing methods and procedures
- Testing tools
- Test data
- Traceability
- Other content items as appropriate to your project
- Deliverables
- Resources
- Timelines
- Risks and contingencies
As you develop the test strategy document, the creation of the document and working through the process of gathering information can help you assess and determine the strategy. The test strategy document can also be useful to convey the strategy to stakeholders to gain their agreement to your testing approach.
Test plan document
We use the test plan document to defines the goals and objectives of testing within the scope of an iteration, or if the project is small, for the entire project. For smaller projects, our test plan and strategy document may be the same document.
The test plan document might capture the following information:
- Preparations
- Staffing
- Test coverage
- Any testing requirements (technical or otherwise)
- Test environments
- Entry criteria
- Exit criteria
- Delegation of responsibilities
- Facility acquisition
- Task planning
- Scheduling
- Documentation on coordination and collaboration with other teams
- Risks and issues that may impact testing
- Specific deliverables of the test project
The purpose is to outline and communicate the details of the testing effort for a specific period of time. It can be used to direct and guide the test effort. It may or may not contain the details found in the test project plan or test estimates (described below). Whether or not those documents are separate or contained under the plan again depends on the size of the project and the context in which we are working.
Test estimates and detailed plans
We sometimes find we need to do formal estimating when we plan our projects. To do this, we like to use bottom-up estimation. Working backwards from our test objectives, we can define all the tasks necessary to complete the objective. When finished, we have a work breakdown structure defined for many of our objectives. We can then take the various work breakdown structures and estimate the work for each of the tasks.
Once we have the estimates for the work, we lay out those estimates against a high-level schedule to understand what our resource needs are going to be. At the end of this process we have estimates in terms of work effort (hours, days, etc.) and resources (a number of people for a period of time).
Depending on the project, this information might be transferred into a formal project plan (most likely using Microsoft Project). This allows us to integrate the various work breakdown structures, consolidate the estimates, assign the resources planned and track the changes to the plan over time as the project unfolds. Maintaining a project plan always has the potential to be a distraction, so be sure that you need that kind of formality or that you truly find it helpful before you make the investment.
Other documents based on your context
Depending on the context you're working in, there might be other documents you need or want to produce. These can include models, lists, checklists, standards, templates, process documents, charters or contracts. For a specific example, Mike once included a site map for a Web site as part of his test planning documents, including details around each of the pages.
There's really no limit to what might be helpful when planning your project. From a documentation perspective, we suggest finding a balance between what's required, what's helpful in driving your understanding of the work, and what's helpful in communicating the testing to the various stakeholders on the project.
-----------------------------------------
About the authors:
Mike Kelly is a principle software test analyst for Liberty Mutual. He is also currently president of the Association for Software Testing and is a co-founder of the Indianapolis Workshops on Software Testing.
Karen N. Johnson is an independent software test consultant. She has 14 years' experience in software testing and software test management and is a frequent speaker at software testing conferences.
