Get started Bring yourself up to speed with our introductory content.

Four tips for effective software testing


Know the correct application results

Source:  Worakorn/Fotolia
Visual Editor: Sarah Evans/TechTarget

Defining expected results independently of and before actual results is necessary but not sufficient. The expected results have to be correct. You have to know the correct application results in order to tell whether the product is producing it correctly.

In general, real business requirements are the basis for determining correct application results. However, too often real business requirements are inadequately identified. Moreover, most testing is based on demonstrating that the product meets its feature requirements, which means the product works as designed. Demonstrating the product works as designed is necessary but not sufficient for -- let alone the same as -- demonstrating that the product as designed satisfies the real business requirements and thereby accomplishes the value it should.

Some exploratory testers believe their main purpose is to ascertain how the software works, essentially investigating many different instances of, "What happens if I try this?" Although perhaps interesting and even sometimes enlightening, this approach is a form of defining actual results that quite intentionally omits consciously determining what the right answer should be.

Ultimately, tests need to demonstrate that products not only work as designed but in fact satisfy the real business requirements, which are the basis for the "right answers" and should include most quality factors. Most developers and most testers, exploratory and otherwise, focus on product requirements without adequately understanding the real business requirements the product must satisfy to provide value.

Exploratory testing is one of many methods to help identify wrong and missed requirements, but it's usually neither the most economical nor most effective means to do so. Simply detecting requirements issues doesn't automatically correct them; and corrections are easily lost or distorted when they are only in the tester's mind. Moreover, I find it unlikely that exploratory testers who explicitly don't want to know the requirements somehow magically can know better, based on typical requirements, what the right answers should be.

View All Photo Stories

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Sometimes there will be an expected outcome, sure, but there is far more to testing than the expected.  In fact the unexpected, the things that aren't tangible static and obvious outcomes and outputs often play a great deal of importance in how a software performs.  (I'm talking things like usability, accessibility, security, and performance).  There is far more to testing than functional checking.
Exploratory testing is one of many methods to help identify wrong and missed requirements, but it's usually neither the most economical nor most effective means to do so.

Effective by what measure? Where's your data to support that conclusion? Or is your evidence purely anecdotal?
I do agree to some extent. Giving some thought to my expected results before beginning hands-on testing definitely helps me to think more clearly and to be less biased. 

As far as the expected results being correct... well, "correct" becomes a bit of a gray area in certain situations. My team supports 150+ applications, and quite a few of them are older legacy applications. Sometimes, no one can tell me what a correct results should look like. That is when I consult with the users and other stakeholders. As long as a result is acceptable to them, then it is acceptable to me.
If one’s results expectations are likely to be wrong, why are they wasting time pretending their testing is providing value? It should be obvious how specious and arrogant it is to presume “I’ll know whether a result is right when I see it,” especially when it’s coupled with active disdain for and almost pride in ignoring whatever requirements have been defined. That said, I agree that defined requirements often, even usually, are woefully inadequate. Inadequacy starts with the fact that they almost always are product/system/software requirements _hows_ which are unlikely to satisfy the REAL business requirements deliverable _whats_ that provide value when met. The primary reason is because REAL business requirements seldom are defined adequately (or often at all), in turn because people (including supposed requirements authorities) think the product requirements are _the_ requirements. REAL business requirements include quality factor “ilities” and other things that “aren't tangible static and obvious outcomes.” They exist regardless whether they’ve been identified. Failure to identify them adds to the likelihood the product/system/software will not adequately address them, if at all. Exploratory’s almost total focus on the software product that has been built dramatically reduces the likelihood of having a clue about the REAL business requirements the product should have addressed, which unfortunately even those who are positioned to know often are not conscious of.

I see numerous discussions were opened about the same article. I think there's a problem with that. I didn't need any predefined expected results to identify the problem.

Users don't think in terms "Gee, the app fails to produce the correct expected results". They expect the product to solve their specific problem and/or bring some value.