The link between automated UI tests and exploratory testing

The link between automated UI tests and exploratory testing

Most agile practitioners recommend a combination of automated user interface (UI) tests and exploratory testing (ET) on agile software projects. But it is a common mistake on agile teams to view UI test automation and ET as entirely separate activities. In fact, each activity should inform the other. In particular, an automated UI test that fails should always be a sign that some ET is required.

How they work
Well-designed automated UI tests are shallow. They should contain very little business logic testing, they should be focused on checking that users have access to page elements that they need, and that users do not have access to page elements that they should not.

Exploratory testing is the process of designing the next test in real time, based on the result of the previous test. ET activity is usually governed by a "charter," some sort of aspect of the system on which to focus testing activities. Examples of charters might be "test the access settings for this feature" or "test that the new feature is consistent with existing features" or "look for inconsistencies in the new behavior."

A failing UI test is an ET charter
Whenever a UI test fails, that means that something in the application has changed in some unexpected way. And where there is an unexpected change, there are often problems to be found. Remember, UI tests are shallow things. They are merely indicators that something in the software may not be correct. It takes human

    Requires Free Membership to View

    Login

    When you register, you'll receive targeted emails designed to keep you informed of the most relevant information on Agile development, application security, testing & QA, software requirements, and more.

    Hannah Smalltree, Editorial Director

    By submitting your registration information to SearchSoftwareQuality.com you agree to receive email communications from TechTarget and TechTarget partners. We encourage you to read our Privacy Policy which contains important disclosures about how we collect and use your registration and other information. If you reside outside of the United States, by submitting this registration information you consent to having your personal data transferred to and processed in the United States. Your use of SearchSoftwareQuality.com is governed by our Terms of Use. You may contact us at webmaster@TechTarget.com.

beings to evaluate the extent of any potential problem.

Responsible agile teams will treat a failing UI test as a sign that some exploratory testing is needed in that area of the application.

Failure modes

Appearance and layout
No automated test in the world can validate that the application actually looks good. When a UI test fails, a tester should pay attention to that area of the application looking for things like skewed text, wrong colors, poor layout, even misspellings. Changing one area of the UI often means that nearby areas become distorted or disrupted. When a UI test fails, always make sure that everything near the element that caused the failure is still nicely formed.

User experience and workflow
Most software applications automate some business function that users depend on. If a UI test fails, that may mean that users will be exposed to a new path through the application. That new path may or may not be well-designed or even correct. When a UI test fails, a tester should pay attention to whether the new experience for the user is an improvement, or whether the change to the software has inadvertently reduced the quality of the experience for the user.

Defects
A failing test of any sort often means that the software is simply wrong. But again, an automated UI test is a shallow thing. The software may be wrong in very subtle ways, or might be failing only after a particular set of actions. The underlying data for the system might be mishandled in some way, or some aspect of the environment itself may be incorrect. Finding issues like this is where ET is strongest, and a failing UI test is often a pointer to such issues.

Turning the situation around
It is also the case that ET may influence how UI tests are implemented. Exploratory testing may discover issues in a part of the application that should be covered by automated UI tests but is not. The automated UI test suite may get too unwieldy for certain aspects of the application and regular ET activity for those aspects may be required. A productive relationship between automated UI test design and ET flows both ways.

Interactions
Both automated UI test design and ET are activities that demand extensive discipline, skill and intelligence. Probably very few testers in the world are expert in both activities. Many people who are good at automated UI test design may actually call themselves "developers." Many people who are good at ET may call themselves "Product Owners" or even "system administrators." Regardless of who is responsible for automated UI test design and who is responsible for ET on the project, it is a mistake for them to ignore the other's work. And it is worth pointing out that experts in both disciplines are in high demand. Being an expert automated UI test designer and an expert at ET is a worthy career goal.

This was first published in September 2010

Join the conversationComment

Share
Comments

    Results

    Contribute to the conversation

    All fields are required. Comments will appear at the bottom of the article.

    Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.

    Back to top