Master core deliverables: Testing Center of Excellence tutorial

• Core ingredients of a Testing Center of Excellence
• Defining, mastering core deliverables of a Testing Center of Excellence
• Choosing, using software, tools for a Testing Center of Excellence

In this installment of my tutorial on building a software Testing Center of Excellence (TCoE), I discuss the core set of deliverables that a TCoE should be responsible for creating and maintaining. We'll examine testing workflows, methodologies and templates, as well as specific artifacts for its partners including: Test Strategy, Test Plans, Test Cases, Test Automation (scripts), Execution results, Test Defects, Test Reports, and all test management artifacts.

The TCoE owns the test architectural vision. This is probably the most critical role for the TCoE and should only be undertaken by senior testing resources. The test architecture represents the selection and integration of the appropriate set of tools, processes, standards and procedures or methodologies to ensure overall testing efficiency.

The formulation of the overall architecture is a collaborative effort between the TCoE Architect, TCoE Manager, TCoE Methodologist, Senior Test Automation Engineers, software vendors, and Quality Assurance. This overarching strategy needs to be captured within the context of a Test Strategy with a supporting architectural schema that supports the immediate and future goals of the TCoE. At a minimum the TCoE architecture should be captured within a TCoE Test Strategy and a TCoE Architectural Schema.

TCoE test methodology: Deliverables

The TCoE owns the test methodology for Integration, Functional, System, and User Acceptance Testing. The TCoE could also own the performance assurance and testing methodology if the Performance Assurance Center of Excellence (PACoE) has been rolled up into the TCoE.

The TCoE provides the process, procedures, and templates that support effective test design and testing deliverables. The Test Methodologist is responsible for these deliverables and should provide education to resources on the testing methodology, working with Quality Assurance (QA) to ensure the methodology is integrated into a corporate continuous improvement program.

At a minimum, the TCoE should provide these deliverables:

• Templates: test plans, test case, test scripts, defects, etc.
• Test process workflows
• Entry and Exit Gates into the overall system delivery lifecycle

The point is not to create an overwhelming set of deliverables. In fact, the objective should be to create the thinnest set of deliverables possible, leveraging every tool available to make this a reality; if an artifact cannot show a substantive return on investment it should not exist.

TCoE test management: Deliverables

Managing a test engagement can be one of the most challenging tasks within the IT space; that having been said, the existence of a TCoE provides test managers and leads the repeatable, predictable processes required to correctly size and implement any testing effort. Perhaps the most important tool that a TCoE provides any test manager or team lead is the collateral to resize the testing effort based on schedule impacts and scope changes.

At a minimum the TCoE should provide:

• A test plan or process in the case of Agile like SDLC's
• A resource plan or model in the case of Agile like SDLC's
• A milestone schedule or model in the case of Agile like SDLC's

Once again, the objective is not to create the ultimate set of planning and management deliverables. Instead, the goal is to answer the question: What is the minimum set of deliverables that will allow the test manager to manage the testing process?

TCoE test cycle: Deliverables

Now, let's look at the entire test cycle as it occurs in a Testing Center of Excellence. Here are the steps or components:

• Test case design: A test case design is not the same thing as a test case; the design captures what the test designer/tester is attempting to accomplish with one or more test cases. This can be as informal as a set of notes or a formal deliverable that describes the content of the test cases before the actual tests are constructed.
• Test case construction: A test case is a sequence of steps designed to test one or more aspects of the system under test. At a minimum, each test case step should include: a description of the action, supporting data, and for verification steps - expected results. The test case deliverable can be captured using a "test case template" or by using one of the several commercial/freeware /shareware tools available.
• Test case automation: The automation of a test designer's test case design - the test automation can be accomplished using one of the several commercial/freeware/shareware tools available.
• Test case execution: Test case execution is the actual running or execution of a test case. This can be done manually or via automated scripts that perform the actions of the test case.
• Capturing test results: Capturing test results is a simple itemization of the success or failure of any given step in a test case. Failure of a test case step does not necessarily mean that a defect has been found; it simply means the application did not behave as expected within the context of the test case. There are several common reasons for a test case step to fail: invalid test design/ expectations, invalid test data or invalid application state. The tester should ensure that the failure was caused by the application not performing to specification and that can the failure can be replicated before raising a defect.
• Documenting defects: The tester documents any defects found during the execution of the test case. The tester captures: tester name, defect name, defect description, severity, impacted functional area, and any other information that would help in the remediation of the defect. A defect is the primary deliverable of any tester – it is what is used to communicate to the project team.
• Test coverage analysis:The TCoE must determine if the testing mandate and defined testing scope have been satisfied, and then document the current state of the application. How coverage analysis is performed is dependent on the sources available to the tester. If the tester was able to map test cases to well formulated requirements then coverage analysis is a straightforward exercise. If this is not the case the TCoE must map test cases to functional areas of the application and determine if the coverage is "sufficient" – this is obviously more of a "gut-check" than a true analysis.

Continue to part three of the Testing Center of Excellence tutorial: Cementing a test architecture.