“Find your mobile sanity,” suggested independent consultant Karen Johnson, presenter at STAREAST 2012 earlier this year. Her session, “Testing in the Mobile Environment: A Functional View,” offered immediate strategies to approach mobile testing with calm focus. The mobile environment is moving fast, but testers can keep up by breaking the process down into workable solutions. Johnson explains some of her strategies in a pre-conference interview.
SSQ: What issues do you see people having the most trouble with in functional testing for mobile devices?
Karen Johnson: One of the biggest challenges is deciding what devices to test, and that challenge is such a preoccupation that it keeps people from focusing and determining what testing they need to do from a functional perspective, or from any other perspective such as security or accessibility.
Another challenge is having a fairly new computing platform where the issues are not well understood or have yet to be realized. If we think back to when the Internet was “new,” it took some time before people had a sense of where the issues were happening and what to do to prevent or test with that knowledge. We need to learn all over again.
What I often see when I teach mobile testing is there is so much focus on getting to a device and getting an application to the device that once those hurdles have been met testers freeze up and are unsure what to do next.
SSQ: So how does a tester go about deciding which devices to test? Some people say it’s important to test not only the newest models, but the previous models as well, as many users still use the version one or two models back. Other people say you should test on all devices that users might be using. What’s your take on this?
Johnson: I ask people, “What is your application that you’re testing— what’s your world?” If you live in North America and you’re testing an application that is never used outside of this country, that already says something, and it says a lot. If you’re American-based, you tend to think everybody’s on a Droid or an iPhone, and anything else is some sort of old, archaic flip phone. That’s our mindset. So then the whole debate is what version of Droid and iPhone are you going to cover. And that’s kind of it. It’s really not a bigger challenge than that. All of a sudden it makes that whole problem space much smaller to solve.
But if you’re working on an application or you have a client that’s in multiple countries, that changes the situation greatly. All of a sudden the whole world isn’t on an iPhone or a Droid. Then you really have to dig in and make decisions based on statistics. My favorite statistics are about the application itself on the device that you’re testing.
The catch-22 is that when you’re deploying it for the first time, you don’t really know what the answer is yet. I start by doing generic statistics—what can we find from companies like CompScore, Global Stats, W3C? And then once your application is rolled out and you can start getting statistics on your own app; then you’ve got better information you can work off of.
SSQ: How does a tester best balance the considerations related to multilingual settings and the different custom settings and tools offered by the various mobile devices out there?
Johnson: Most of the testers I’ve met are not working with multilingual applications, but they are challenged by learning devices with so many settings and those settings being buried in different places and menus on the devices. What settings are relevant and important to an application are important to learn first – unfortunately it’s not obvious.
For testers like myself who have had to address multilingual applications, there is an extra challenge; sometimes even if you can change a device to another language, it doesn’t mean the phone keyboard is ready to handle all the languages you might need to work with.
SSQ: In your view, what security concerns are the most crucial?
Johnson: Mcommerce -- credit card processing, banking applications -- any situation where money is being processed or credit cards are being used. We were anxious with the Web and now we are anxious with mobile, and with good reason. We have issues yet to understand, we have a less secure platform and we have money to lose as well as company reputation and customer loyalty and trust. We need to close that gap as fast as we can learn how to.
SSQ: What do you anticipate attendees will take away from your session at STAREAST?
Johnson: There are four things I’d like people to pick up:
1. Find your mobile sanity. I plan to outline areas where people already have skills and an existing background in testing and to recognize that while there are many new aspects to learn and consider; don’t get overwhelmed.
2. Consider the “problem space.” Mobile devices will not be used from inside perfectly connected offices with fully charged phones, we need to reconsider how we build and manage a testing team. Let people out of the office; go mobile to test mobile.
3. Gain some immediate ideas. I plan to highlight some areas in mobile such as functional and user interface considerations that testers can walk out the door from the session and immediately begin testing.
4. Find resources for ongoing learning. So a one-hour session is nothing compared to trying to school yourself in new computing platform, where do you go to keep learning? I highlight different ways to learn from books, articles, but also Twitter as ways to find statistics to keep up with.
For comprehensive conference coverage, see our Software Testing Analysis and Review conference page.