How to define a test strategy

Digg This!     StumbleUpon     Del.icio.us

RELATED CONTENT Software Testing and Quality Assurance What are the top free and not-free automated test tools? Test case preparation for a Web-based application The difference between functional testing and regression testing Free load/performance testing tools for Java-based Web applications Mapping results in test environment to production environment What are some debugging shortcuts to reduce frequent debugging? How to get a software testing job in a recession How to write a test strategy document What are the risks in using open source test automation tools? What is system integration testing? Software testing and quality assurance (QA) fundamentals Five roles test managers play in agile development: Tutorial, part one Three software regression testing steps can perfect defect fixes Software Testing: Assessing risk and scope Software Testing: How to know you're ready to start testing Quality assurance (QA) and testing's role in requirements Test case preparation for a Web-based application The difference between functional testing and regression testing Mapping results in test environment to production environment Agile software development tutorial: Agile testing Creating strong QA and testing strategies in a changing world Software testing models and approaches (Context-driven, Factory, Analytic, Quality, IV&V) Five roles test managers play in agile development: Tutorial, part one How test managers can shine in agile development: Tutorial, part two Sam Charrington, Appistry CloudIO Testing mobile Web applications for usability and context Software Testing: Assessing risk and scope IBM strengthens ALM line, as Micro Focus buys competitors Borland, part of Compuware Using session-based test management for exploratory testing Agile software development tutorial: Agile testing The benefits of exploratory testing in agile environments What are the different software testing methodologies?

RELATED GLOSSARY TERMS Terms from Whatis.com − the technology online dictionary build  (SearchSoftwareQuality.com) code review  (SearchSoftwareQuality.com) conformance testing  (SearchSoftwareQuality.com) error handling  (SearchSoftwareQuality.com) garbage in, garbage out  (SearchSoftwareQuality.com) load testing  (SearchSoftwareQuality.com) NUnit  (SearchSoftwareQuality.com) quality assurance  (SearchSoftwareQuality.com) stress testing  (SearchSoftwareQuality.com) white box  (SearchSoftwareQuality.com)

RELATED RESOURCES 2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems Search Bitpipe.com for the latest white papers and business webcasts Whatis.com, the online computer dictionary

This is a broad question with several possible meanings. However, I'll take a stab at it. It sounds to me like the question is how do you drive a system test strategy, i.e., once each component of a project has been component-tested, how do you test the strategy? What do you need to know from developers to build the strategy?

When it comes to system testing, to be frank I want less information from the developers than I do from the customers. I want to approach my system testing from a scenario basis. That having been said, there are some important things to know about the components -- specifically, how they interact with each other. What are outbound and inbound dependencies (i.e., what data is transferred between components)? The key here is to ask questions that don't offload the burden of testing from you to the developers. You can't ask a developer, "How do I test this" because, if he answers you, he might as well do it himself. What you need to ask are questions such as "How does this component interact with that?" or (better yet) "I've been reading the technical specification for your component, and I have a couple of questions." Then ask your specific questions.

To put it into a real-world analogy, let's say you're testing a procurement and inventory control application for a small gas business. The application may consist of a procurement piece (code that automates ordering delivery of gas), a projection piece (code that projects short- and mid-term inventory needs), and a delivery tracking piece (code that verifies the gas ordered is delivered, even if it's split up among several deliveries). At this point in your test planning, your interview with developers will focus on the data being shared --which portions of the database are common and which are specific to a given component. You'll also ask how components modify shared data. While this data modification may have been tested to specification during initial testing, it's very possible that the original specification overlooked some element of interaction and the spec is deficient.

Software testing resources: The difference between a test plan and a test strategy Ten skills of highly effective software testers Ten software testing traps

Another key step is to examine the test strategy for each individual component. Here you are looking for the overlap: Which cases do two or more components have in common? Often, the team developing the integration test strategy will have spent time identifying system-level test cases, and you can leverage them here.

In our real-world example, interviewing the test leads from each team should result in them sharing with you cases that they felt were "system test cases" by nature -- cases that cover interaction, cases with cover dependencies, etc. The test lead for the procurement component, for instance, might have identified cases that cover order size and delivery date and will want the delivery tracking piece to be sure to check that they handle split orders appropriately. Through these interviews, you should build a list of cases that cover common cases.

Finally, as I mentioned, you want to spend a lot of time in system-level testing thinking about scenarios. You want to define how a user will interact with your product and follow that interaction as it goes from component to component. You definitely want to speak with the customer and in two phases. First, sit down with the customer and ask them to work with you to identify key customer scenarios. Document everything! Then, from that meeting, develop any other scenarios or obvious variations on scenarios. Prioritize these scenarios and write up the steps that comprise the scenario. Finally, return to the customer and validate your final scenarios and your steps that cover them.

In our real-world example, you'd walk through the lifetime of an order -- from the moment the projection component identifies a new order is needed, through order placement and then fulfillment.

You definitely want to script scenarios that cover full order delivery as well as split deliveries. You want to run a scenario that probes how the projection component deals with fluctuating demand, and so on. Once you've identified a set of scenarios, script high-level steps for them. Circle back, refine your scenarios and steps, and then sit with the customer and have them validate your planning -- taking feedback and modifying appropriately.

Through this research, you can begin to identify a system-level test approach that results in the highest possibility of verifying customer functionality. You'll minimize test case overlap with your integration-level testing, as well. The key to good system-level work is focusing on higher-level testing (scenario-based testing) and minimizing your component-level testing (assuming component-level testing has been carried out successfully). If you involve developers in this stage, be sure to do so as a planning augmentation. Don't ask them to define your strategy for you. Bring them in as valued experts, but be sure to minimize the questions you ask them. You want them on your side when it comes time to advocate for fixes.