STPCon Fall 2013 calls all software test professionals to Phoenix
A comprehensive collection of articles, videos and more, hand-picked by our editors
"The definition of application quality has shifted." This was the thrust of the keynote presentation at STPCon in Phoenix on Wednesday afternoon. John Montgomery, the vice president of product delivery at uTest, expounded on the differences between today's mobile apps and yesterday's Web applications. As technology opens up more and more direct communication channels, savvy mobile developers are getting more customer-focused and taking fewer cues from internal designers.
There was a time when software companies could tell users what quality software was. The software development company had so much sway that they could dictate what the right way to use the software was and what the right machines to run the software were. Ad men like TV's Don Draper could push the software's internal requirements onto the user.
The age of inside-out thinking is done. Long live 'outside in.'
vice president of product delivery, uTest
Today, the situation is very different. More options are readily available to the user. The barriers to switching software are much lower. And today's users are much less tolerant of spotty quality than those in bygone days.
In addition, today's users have much easier methods to share their opinions of the software they use. A user can immediately post a review to the app marketplace, letting other users, as well as software developers, know what works and what doesn't work. Montgomery pointed out that better reviews are tied to higher download rates, and higher download rates mean more success.
More feedback is a good thing
Montgomery labeled the old way of looking at the software quality process "inside out," meaning requirements and design decisions start inside the development organization and are pushed out to the users. He said, "The age of inside-out thinking is done. Long live 'outside in.'" By 'outside in,' Montgomery referred to the customer-focused process of pulling in user requirements and other feedback from outside the organization in order to make informed decisions.
"User perception is our reality," according to Montgomery. He pointed out that mobile app consumers read app store reviews before they purchase or download new apps. If the reviews are generally bad, a user is likely to move on to an app with better reviews. "And the only way we can affect the reviews," he said, "is to produce a product they want to write a good review about."
"Users -- and all humans -- love to tell you what they think," Montgomery said. "The question is, 'How do you listen?'" He said that it's important to use as much explicit data as possible in application design. User reviews are a great source for that data, but it's all unstructured -- which can make it difficult to harness.
Track the trends, not the issues
Montgomery suggested keyword analysis as a really helpful tool to help project managers focus on the overall trends. Looking at reviews anecdotally, he warned, might get project managers wrapped up in individual issues. Tracking how many times user reviews include words like crash or slow or ugly against the number of stars they give the software reveals a lot.
If the one-star ratings almost all include the word crash, for example, while the five-star ratings rarely include it, that might be an indication that the application doesn't work on a number of devices or operating systems. Gathering metadata around crash reports can likely reveal which ones are having problems. If reviews for the same application include the word ugly in about the same number of good reviews as bad reviews, it might mean that visual design isn't very important to the user base.
Montgomery made it very clear that he believes in-house testing is essential to enterprise development efforts. Enterprise development projects need customer-focused test professionals that understand the particulars of our application architecture. However, customer-focused mobile application development also requires a broad set of testers with expertise in testing a wide variety of devices. Montgomery argued that outsourcing the device testing frees up internal testers to do more in-depth research on a mobile app's core functionality.