Mobile embedded applications are more vulnerable to attack and more difficult to test than most mobile applications....
They are widely used in such regulated industries as health care and finance, where data loss is really risky. Compounding the risk is that many of these mobile embedded apps are designed to continuously transmit confidential data over hard-to-tame Wi-Fi networks. That makes them easy targets for skilled attackers seeking to intercept private information, and it also requires quality assurance pros to write custom tests to secure that information.
Because mobile embedded apps are designed to for industry-specific use, they also require that testing teams engage subject matter experts to conduct code reviews. "There are so many factors to deal with," said Chris Rommel, vice president at VDC Research Group Inc. It's challenging enough to make sure that vertical-specific, mobile applications do what they are supposed to do from a functional standpoint, he said. "And on top of that, you have to protect data, and ward off attacks."
In this article, experts outline seven challenges in testing mobile embedded apps.
There's a wide range of devices, processors and operating systems.
The first thing mobile testers must consider is the sheer diversity of the platforms they are dealing with, Rommel said. "You have to take into account all the different types of machines, processors and operating systems." Many mobile embedded apps still run traditional real-time operating systems, and that means there are literally dozens of combinations out there, he said. "That makes it harder to predict what to test for."
Mobile embedded applications transmit data over wireless networks.
A bigger hurdle is understanding the environment in which a mobile embedded app will be deployed. In the past, embedded mobile applications often operated as standalone devices. Today, however, these devices are likely to be part of a larger, connected enterprise, Rommel said. For example, in health care, it's not uncommon for disparate devices to collect information from and serve a single patient. That information is continuously uploaded across wireless networks to larger hospital systems. That makes the data vulnerable to wireless network attacks, he said. "You have to establish robust testing practices to address these threats," he added, referring to source code analysis and penetration testing, two techniques for security testing.
Code review is a manual, time-consuming process, and it cannot be overlooked.
Mobile embedded applications often can be exploited at the hardware level.
Source code analysis and penetration testing are best practices for all applications. But mobile embedded apps can usher in additional security concerns if the application makes calls to the hardware level, Rommel said. It's common for applications, such as those used for medical imaging, to do that in order to enable faster processing, he said. "In each hardware architecture, there are small bits of abstraction, small assumptions that can be exploited."
Mobile embedded apps are often subject to compliance mandates.
In regulated industries, mobile embedded applications are commonplace. For example, medical devices must meet FDA requirements and also comply with the Healthcare Insurance Portability and Accountability Act (HIPAA), said Rommel said. HIPAA stipulates, among other things, that patient data must be kept private. You have to make sure that data is protected, and offer proof of how you protected it, he said.
Quality assurance pros working with mobile embedded apps must write their own tests.
If you are testing generic software on a PC, there are plenty of prebuilt automated testing environments available to help, said Gwyn Fisher, chief technology officer for Klocwork Inc., which sells source code analysis and other developer tools. "But with embedded mobile apps, ideally, testers have a development slant," he said. "With embedded apps, where proprietary frameworks are commonplace, you are not going to get off-the-shelf automation testing tools."
One way to get help with this process is look at the simulation engines provided by the platform providers for embedded systems, said Wayne Ariola, vice president of strategy for Parasoft Corp., which sells test software. "The simulators do a lot of electronic stuff that software guys don't care about. But they also simulate how software should interface with the hardware," he said. "You can use this information to develop some tests."
Embedded apps require code reviews from subject matter experts.
In addition to writing your own tests, it's also essential to engage subject matter experts to conduct code reviews, Klocwork's Fisher said. "You have to make surethe embedded app does what it is supposed to do." Code review is a manual, time-consuming process, and it cannot be overlooked, he said. "Tools can vet algorithms. But they cannot replace code reviews," he added. "You have to find out if this app is going to do bad things -- or is it going to do the right things?"
Embedded apps raise expectations for developers and testers.
In embedded-application development projects, professional developers must take responsibility for checking in code that works for the lifetime of the device, said Fisher said. "It's a more responsible role than it has ever been in the past," he said. "You have to prove compliance and prove you did static analysis," he added. "You have to show me that you are doing the right thing."