With the ever-growing numbers of devices and the applications that are being written for use on those devices,...
the demand for mobile testing is huge, said Doron Reuveni of uTest at STAREast recently. But how can an organization test with such a large number of configurations? Are they to purchase every mobile device to check for compatibility? One answer he suggests, is crowdsource testing.
Crowdsourcing is not outsourcing or offshoring, but instead, "leveraging a community of customer- related professionals" to perform a task, said Reuveni. This article shares his advice on the uses of crowdsourcing and information about crowdsourcing projects at uTest, gleaned his session and a SearchSoftwareQuality interview at STAREast in Orlando, Fla.
Reuveni's favorite definition of crowdsourcing is paraphrased from Jeff Howe, journalist and author of Crowdsourcing:
"The act of taking a job or service traditionally performed by a designated person (usually an employee) and sourcing it to a large group of people in the form of an open call."
Mobile applications are no longer simply games and music, Reuveni said, but are expanding to include line-of-business applications: CRM, ERP, POS and HR systems; productivity applications; document editing and expense tracking; collaboration applications: email, IM and social media; and security applications: VPN and anti-virus. With the increased use of mobile devices for mission critical applications and access to customer and financial data, there is pressure to ensure mobile applications work securely across multiple operating systems, handsets, languages and locations.
"The applications are starting to drive the devices," remarked Reuveni, underscoring the fierce competition between mobile developers. Clearly quality is increasing in importance as more critical applications are deployed on mobile devices.
However testing on mobile devices has obstacles not encountered with traditional applications. The testing matrix becomes increasingly complex, time-to-market is more critical and testing by location is important.
Reuveni described crowdsourcing as a method for building an "on-demand testing team" reflecting the diversity of the user group and applications. Furthermore, the applications can be tested in the actual locations where they'll be deployed. Though Reuvini said that crowdsourcing is not the one and only solution to mobile testing, he said it was a "critical piece."
Reuveni went on to describe how crowdsourcing works at uTest. Over 25,000 testers in more than 160 countries are in the uTest community and have each filled out a profile indicating the software and hardware that they have available. As projects come in, depending on the needs of the customer, profiles are scanned to find those testers that have the right skills and resources to contribute. These testers are then invited to join the project. They become part of a project team that then performs test on the applications.
The testers are paid based on the bugs they find and the reports that they generate. They earn "reputation points" and as their reputation improves, they are paid more, Reuveni said.
Another provider of crowdsourced mobile device testing is Mob4Hire. The site allows a way to match developers to freelance testers. Developers register and can describe a "Project" that they need tested, according to Reuveni. Testers can register and describe the handsets that they have available to test. Mob4Hire will send notification of projects to the testers that meet the requirements and those testers are able to bid on the project. Once the project has completed, developers pay the testers after a five-day check for quality, with a commission going to Mob4Hire. Both developers and testers are rated after each test, which can help in the decision-making process for future projects.
In a post-session interview, Reuveni talked more about uTest and the newly-launched Version 3 of the website.
"It was tested quite extensively with our own community of professional testers. It's a completely new user interface," Reuveni said. The new version has a completely open architecture, so the next phase is to integrate it in with many platforms from various locations. You can work with uTest from any location around the world. " Reuveni said integration with bugtracking tools and test case managements systems is also in the plan "A lot of new features and functionalities are going to come out in the next 3 months or so."
Whether you work for uTest, Mob4Hire or a different vendor, crowdsourcing appears to be a new trend in the mobile test market, Reuvini said. Crowdsourcing is not the one and only solution to mobile testing; however, he said, it is a "critical piece."