Here in the 21st century, the testing of business function is handled more and more by machines. Test automation...
at every level is more and more prevalent, and the design is improving to allow testing with greater ease than the software world has ever experienced before.
The role of the dedicated tester is changing as test automation improves. Some dedicated testers are becoming experts in automation; other dedicated testers are becoming experts in test design and Acceptance Test Driven Development (ATDD).
But there is another role for a dedicated tester that is being neglected and overlooked in the face of increasing test automation: the user experience tester.
The neglect of aesthetics
Compared to other ubiquitous products in our lives like furniture and silverware, computers have not existed very long. And for most of the history of the computer, interfaces for using them have required a certain amount of expertise and training. First there were toggle consoles; then paper cards; then green-screen terminals with static input fields and single-color displays. Only in the 1980s did computers for consumers acquire folders and menu bars and a mouse.
But much of our testing practices have roots in the earlier incarnations of the computer, where the concept of a 'defect' had to do either with incorrect function of the internal state of the computer; or else with incorrect presentation of data to the user; or else with incorrect handling of data submitted by the user. Test automation checks for this sort of error very well, and today is relatively easy to put in place.
While the interface presented to computer users has become almost unimaginably complex since the first Graphical User Interface (GUI) in the 1980s, our ideas about testing GUIs are still geared to testing the function of the application.
There is particular interest in the testing of Web applications, since the Web is clearly the future of commercial software applications; Google Chrome OS is an entire operating system devoted to running only a browser; Google maintains that the majority of computer users need only a browser.
Tools like Selenium and Watir make it relatively easy to test functions exposed by a browser. But treating browser testing with such tools as the end of the test effort is a dire mistake.
Commercial applications in order to succeed, not only have to function correctly; they have to provide the user with an experience that the user wants to experience. Applications that fail to delight their users will ultimately fail in the marketplace, and are replaced by applications that provide the same function with a more pleasing UI.
Development teams seldom test an application for its ability to please users. Such testing has not been important in the history of computing until very recently. Now such testing is critical and we have few tools with which to accomplish such testing.
The application of aesthetics in testing
There is a field of software design called "User Experience (UX) Design". According to Wikipedia, UX Design incorporates "aspects of psychology, anthropology, computer science, graphic design, industrial design and cognitive science."
Software testing is evaluation. Evaluating UX, informed by such a disparate set of disciplines, becomes a difficult task. And if the only tools on hand to perform such testing are functional tools, then testing UX might be impossible.
But we can use other tools from other disciplines to do this testing.
We desire to test for a good user experience and for a bad user experience, or a delightful user experience and an appalling user experience or a pleasing user experience and an annoying user experience. These adjectives are all fundamentally aesthetic judgments. We need to explicitly apply tools based in aesthetics in our testing.
A set of aesthetic heuristics
Any particular set of aesthetic heuristics will almost certainly rely on context, but as an exercise, here are some possible examples:
- Consistency: the user should never be confused about his position in the application. Controls that perform similar functions should have similar appearance. Related pages should have a similar appearance.
- Harmony: Color schemes in the application should not distract the user. The composition of pages should contain enough controls to be functional but not so many controls as to be hard to identify any particular control.
- Unity: Each element of the application should support every other element. No part of the application should be superfluous.
- Variety: The application should provide the user enough different functions to be effective.
- Simplicity: The user should be presented with the simplest possible interface to accomplish his objective. The Google search page is a canonical example: it has one input box and two buttons.
These are just examples of the sort of aesthetic criteria we might use to judge user experience.
Looking to the future
Our programming languages are becoming more and more abstract, farther and farther removed from the 0s and 1s actually controlling the computer hardware. Our applications are becoming more and more abstract, being hosted in a cloud of ubiquitous computing power. Our software development is becoming more abstract, as we spend more time simply connecting libraries and APIs to create applications rather than coding our infrastructure from scratch. The principles the Computer Science discipline borrows from Engineering and Manufacturing serve us more and more poorly.
As this process of abstraction continues, and as it becomes more and more imperative to supply our customers and clients with an aesthetically pleasing experience, we as testers have look further afield for the tools to evaluate applications that provide that User Experience. Successful UX testers will invent tools based in art, literature, and psychology. Let us start now.