Jezper - Fotolia


The secret to asking the right usability testing questions

In the brave new world of software testing, usability can be overwhelming. Expert Gerie Owen guides us through asking the right usability testing questions.

Testers live in a digital world, and in this world customer experience is the key to success. J.D. Power, an organization that studies and rates customer satisfaction across a wide range of products and services, focuses heavily on Web customer experience. Digital customer experience on the Web impacts more than 50% of the customer satisfaction areas included in the overall score. So, how does an organization ensure a high-quality digital interaction with their customers? Usability is the key. Testers, it's a brave new world. Here is how to ask the key usability testing questions.

Although usability testing offers many new challenges and opportunities, the most significant and complex challenge is also the most basic. This is the challenge of understanding the user. Testers must understand the intentions of the customer and their expectations in order to address the vital usability testing questions.

Because usability testing is a core component of mobile testing, it is important to understand the mobile customer and how, where and when the customer will interact with the application. This also includes an understanding of the environmental factors involved in the interaction. In addition to environmental and context considerations, connectivity, including data transfer speed, downloading time and the varying speeds of wireless networks add new challenges.

The scope of mobile usability testing questions expands beyond device/platform/browser combinations. Form factors including variation in screen size, display resolution, and method of navigation, as well as data entry add complexity to usability testing. The logistics of testing, including decisions on actual device versus emulation, field versus on-site, and whether or not to bring in actual potential users present new challenges.

Finally, in our objective world of testing, how do we measure something as inherently subjective as usability? What are the criteria on which our usability tests pass or fail?

How do we, as testers, approach these usability testing questions? First and foremost, we must test early and often. More than any other component of application development, usability is made or broken in design. Poor design choices that become evident during testing are complicated and expensive to fix; therefore, it is critical that testers work closely with the usability designers.

Field testing has the distinct advantage of providing the opportunity to test in context.

Our test strategy and plan for usability testing, like any testing, should include what we will test and how we will execute the testing with a focus on the users and their interaction with the application. Determining what we will test should be based on our understanding of the customer.

So, how do we learn about our customers? Working closely with usability designers, testers can learn about the customer through personas and user value stories. Personas are archetypical users who represent the motivations, goals and expectations for interaction of their respective user groups. User value stories are scenarios in which the user interacts with the application and achieves value or not. Personas and user value stories are powerful tools for developing user-centric test scenarios.

Personas and user value stories provide a basis for our test scenarios; however, we also need criteria for evaluation. D. Zhang, in Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, suggests nine attributes that are based on ISO 9241 as well as human-computer interaction design standards. They are learnability, memorability, efficiency, errors, effectiveness, simplicity, user satisfaction, comprehensibility and learning performance. Overall, these are measures of how users learn, understand and remember how to use the application, as well as how efficiently and effectively they accomplish their intentions and derive value from the application.

Now that we have our test cases, how, where and when do we execute them? And who executes the test? There are several methods of usability testing. Creating an on-site test lab offers the ability to control the test coverage and it provides measurable results. Using a test lab also provides the option to moderate the test. In this scenario, the test moderator sits with the testers, assists and documents issues as the testing progresses. In an unmoderated scenario, the testers execute the tasks on their own and share their observations to the tester at the end of the test period.

Field testing has the distinct advantage of providing the opportunity to test in context. Jonathan Kohl, founder and principal software consultant at Kohl Concepts Inc., suggests that UX designers should "use the real world as your user interface." Usability testers need to include field testing not only to cover environmental context scenarios but also evaluate their user value stories in action.

What is the most important skill that testers need for effective usability testing? It is the ability to understand the needs, intentions and values of the customer and thereby being able to develop effective usability testing questions. If you are focused on the user throughout the test cycle, you will be testing usability inherently. Testers, enjoy the brave new world of usability.

Next Steps

Does your mobile app cut it? Here's how to find out

Sometimes it takes a village (or a crowd) to test

The case for outsourcing testing

Finding the value in customer experience testing

Dig Deeper on Topics Archive