The explosion of data traffic over wireless networks
Cisco’s global estimates indicate that the mobile traffic has increased by over 160% during the period 2008 – 2009. The Cisco VNI Mobile 2010 report indicates that mobile data traffic is expected to grow by 39 times between 2009 and 2014. These trends clearly indicate end users comfort in using smart phone applications to perform day to day activities. This shift in consumer behavior is putting significant pressure on organizations to anchor business applications, both old and new, on mobile platforms.
The biggest challenge in using applications on a mobile device is the dependency of its performance on an unreliable wireless communication network. Novarum’s report on 3G speed and reliability suggests that 3G networks in a US city in 2009 had unreliability as low as 52%, marginally improving to 55% in 2010.
The exponential increase in data traffic, coupled with the unreliability of networks, poses connectivity challenges to mobile applications that work in a client-server mode. Organizations that intend to adopt the mobile platform as a key business strategy need to ensure that every single mobile application developed undergoes rigorous test scenarios that take into account the “variability” on a mobile network, besides the other important performance factors.
Causes of network variability
Network variability refers to the changing behavior of a wireless network. The variability is caused by the following factors:
Signal loss: Dead spots on a wireless network lead to a partial or total loss of network signal. This leads to abrupt snapping of the client server communication.
For example: When a user is transferring funds via a mobile application and there is signal loss/disconnection in the middle of the transaction, even though the fund has been transferred the application might show a “transaction failed” message to the user. This primarily occurs due to the inability of the system to detect the disconnection and handle it properly.
Data loss over network: A data loss occurs when data packets sent by client/server over a wireless network fail to reach the intended destination because of network congestion. Even though data can be retrieved with a mobile operating system, the data loss has a severe impact on the user’s experience, when the application doesn’t handle the resulting error conditions properly.
For example: Data loss over network can result in an application screen freeze, an abrupt application closure or a temporary disablement of some features.
Bandwidth: The bandwidth can fluctuate significantly when multiple users are sharing the same network.
For example: A slow bandwidth would have an impact on a streaming video with problems like buffering.
Enterprise-carrier network switching: With most enterprises Wi-Fi enabled, when an employee comes within the range of his employer’s Wi-Fi network his mobile device switches to a Wi-Fi network from a carrier network.
For example, if a network switch is not handled properly by a mobile application then the enterprise database may get corrupted by the application.
Network delay and jitter: The time taken by each packet from a source to the intended destination is defined as network delay. The network delay experienced by each packet can vary and is known as jitter.
For example: A huge delay and jitter on the wireless network can result in a slow data transfer rate. The mobile application can indicate an error quickly or the user interface of the mobile application can become unresponsive.
Impact of network variability on mobile applications
The variability of a network can have the following impacts on mobile applications:
- Unpredictable application behavior: The mobile application might freeze, abrupt application exit, inconsistent application behavior or irrelevant error messages might occur due to the inability of the mobile application to handle the variation in the network.
- User interface related errors: Issues like images not displayed or the disorientation of user interface display can be caused by bandwidth variation and signal loss. See Figure 1 for a depiction of some user interface issues faced on a leading mobile handset due to network variation.
Figure 1: Snapshot of user interface issues of a leading mobile handset
- Database corruption: Besides the network failure scenarios, IT managers also need to ensure that transactions originating from mobile clients and updating server databases are atomic processes. Database corruption can occur at the server-end due to a network disconnect caused by a signal or packet loss during a transaction, when the client and server databases are not in sync.
- Performance challenges: The performance of the mobile application for all client-server communication scenarios needs to be tested under various network conditions. A low bandwidth or delay/jitter on the network can significantly affect the performance of the mobile application. The application might take more time in completing a transaction or freeze under these network scenarios.
- Functional issues: Various network conditions can lead to synchronization issues between the networks, user interface and data layers of a mobile application, which can have an impact on the functionality of the mobile application.
Drawbacks of field testing (live network testing) on mobile applications
The common drawbacks and challenges of live network testing on mobile applications are as follows:
- Inability to reproduce the live network conditions at a later stage
- Incomplete test coverage due to lack of control over network parameters
- Time consuming for test engineers with the need to travel to multiple locations for testing
Network impact testing solutions
A testing solution that would assess the impact of network variation on a mobile application’s functionality should have the capability to create a wide range of mobile network conditions in a controlled lab environment. It also needs to be flexible to test with an actual device or an emulator, without any change applied to the mobile application, device, server applications or server configurations. Further, the testing solution needs to be able to cater to thin and thick clients.
Some of the typical issues encountered during the field testing of mobile applications are:
- Abrupt application exits
- Premature application timeouts
- Application freeze
- Inconsistent application behavior
- User interface issues like the disorientation of display or images not being displayed
The network impact testing solution needs to be able to address all these issues by recreating these scenarios in a lab environment. Some of the desired features in a network impact testing solution are:
Network disruption testing
|The ability to validate the mobile application functionality under network conditions like connectivity loss.|
|Network performance testing||Emulate real-life network conditions like bandwidth fluctuation, network type (GPRS/EDGE/3G) and also validate the end user experience under these conditions.|
|Network congestion testing||The mobile application validation needs to encompass scenarios of partial data loss and targeted data loss during critical transactions|
|Multi-network testing||The capability to validate the mobile application’s behavior and network performance during a switch in the type of network For e.g. 3G to WLAN or 3G to EDGE and|
|Trace testing||Lastly, the network testing solution should be able to test the mobile application with network traces collected from a live network environment.|
Today, businesses using mobile applications realize the importance of network testing to ensure robustness and functional performance in varied network conditions. If network variability is not addressed, it can have a severe impact on the end user experience which can further have an adverse influence on the brand image and reputation of the enterprise that built the mobile application. The key is to include network impact testing in the overall test plan of the mobile application, in addition to functional, user interface, performance and regression testing. This would ensure high mobile application performance irrespective of the network conditions.
Follow us on Twitter at @SoftwareTestTT and let us know what you thought of this article.
About the Authors
- Manik Jandial (firstname.lastname@example.org) is a Technology Architect with Infosys, with 11 years of experience in mobile applications development, embedded middleware development and testing on various platforms like J2ME, BREW, Windows Mobile, Android. He has designed, developed, tested and ported enterprise as well B2C mobile applications for multiple clients. Manik holds a Bachelor’s Degree in Electronics Engineering from Pune University and a Post Graduate Diploma in Advance Software Design and Development from the Electronics Research and Development Centre of India.
- Arun Somasundara (email@example.com) is a Research Scientist with Infosys Labs. His research is focused on developing tools for mobile application testing. Arun holds a PhD degree from the University of California, Los Angeles (UCLA). His thesis was on how to improve system performance of wireless sensor networks using controlled mobility.
- Karthikeyan (Karthikeyan_Dhanapal@infosys.com) is a Senior Product Manager with Infosys. He has over 13 years of experience in designing and building wired/wireless communication systems. He holds a Master’s Degree in Electrical Engineering from Indian Institute of Science, Bangalore. In his current role, he leads the mobile application resting solutions group at Infosys Labs. His interests include building innovative tools that address critical gaps found in mobile application testing.