Problem solve Get help with specific problems with your technologies, process and projects.

Conducting automated testing on the user interface early and often

Early user interface automated testing can be tricky when the requirements are still changing. Yet, it is important to conduct repeatable testing even when functional and nonfunctional requirements are still stabilizing. This expert response offers an explanation of the advantages of UI automated testing and some tips for doing it effectively.

I know of some groups that are creating user interface automated tests when the requirements are still changing. It seems like it would be better to create the UI tests that require recording after the code is stable. Would you agree?

“Test early and often” is the mantra of savvy analysts, developers, and testers. They know that acting on early feedback is less expensive than doing rework on last-minute feedback. UI automated testing (i.e., record/playback) provides a way to conduct repeatable testing of whether functional and nonfunctional requirements have been met. Yet there are costs to setting up automated testing. And it is very expensive to maintain automated functional tests if the tool you are using fails tests when minor interface layout and navigation changes are made. For example, some automated testing tools will fail a test if the position of a label on a GUI or webpage is even slightly changed.

From the perspective of functional requirements, here are some tips for early automated UI functional testing:

  • Use automated UI functional testing when you have stable functional requirements, or when your UI layout and navigation are stable (even if your functional requirements are still volatile).
  • If your UI layout and navigation are still volatile, identify ways to isolate the UI from functional testing.
  • If the functionality you are testing will be used infrequently, consider whether the benefits of the automated test outweigh the cost of setting it up and maintaining it.
  • Conduct automated regression tests as often as possible, and not just as part of the final QA activities or just before executable code from multiple projects is put into production.
  • Keep in mind that there are many valuable early forms of manual testing. These include:
    • Usability tests, such as card sorting or paper prototypes, that explore navigation flows  with no underlying functionality
    • Reviews, during which someone with a tester’s mind-set evaluates the requirements
    • Having an expert who did not specify the requirements arrive at a solution for a sample set of input data (this is especially useful for complex calculations)
    • Manually testing executable code with yet-to-be-added code sections stubbed out

For nonfunctional testing, UI scripts are very powerful for performance, load, and stress testing (e.g., simulating 1000 customers trying to place an order at the same time). For early testing, if some of the requirements are not completely settled or if you haven’t yet implemented communication with other software components and services, then you can stub out portions of the test. Early performance and load testing provides early warnings to capacity planners and infrastructure architects about any problems with the scalability of the architecture.


Dig Deeper on Automated and autonomous testing

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

Repeatable and frequent testing is important, but UI tests can be so flaky and brittle. This is one of the reasons why we have never maintained UI tests long term.