Cloud-based applications are becoming more common these days, whether they are on private clouds, public clouds or a combination of both. Testing for cloud-based applications presents its own specific challenges. Understanding how these applications are structured goes a long way in designing and executing appropriate test plans for them. These tests are in addition to the usual unit, system and performance testing you would need to perform on any software application. The following challenges exemplify what these additional dimensions of cloud-based testing are and how to address them:
- Browser testing: Browsers are not necessity in cloud-based software applications if thin-client software is used. However, in most cases, a variety of popular browsers are used, all in almost equal measure. It is imperative to list at least the top four or five browsers by global usage, and ensure that the application test plans include testing the applications with these browsers. Scripting at the browser end usually causes browser incompatibilities, and cloud-based applications need to be designed with minimal or no browser scripting if you expect a variety of browsers to be used.
- Mobile platforms: Smart phones and tablets are rapidly joining the ranks of laptops and desktops where cloud-based applications are accessed. Many organizations are happy with mobile Web versions of the user interface for
- cloud-based software applications. However, mobile native apps could also be used instead of accessing the software solution using a mobile Web version. Test plans for native app interfaces need to be drawn up in addition to the browser based tests.
- User interface testing: User interface testing in cloud-based software applications could be the same as in the case of non-cloud-based applications. There are no additional challenges posed by cloud usage but user interface testing needs to be an essential part of the test plans, especially when testing in multiple browsers.
- Performance testing: Cloud-based software applications are used in cases where you have a significant peak performance profile that is more challenging than the average performance profile. For example, a shopping website experiences average performance requirements during most of the year, but the Christmas shopping season may require peak performance requirements from the same application. Simulated peak performance tests need to be added to average performance tests for a more comprehensive, and reliable testing.
- Service provisioning/de-provisioning testing: Private clouds, public clouds or combinations of both could be provisioned when demand increases and de-provisioned when not needed. This provisioning and de-provisioning needs to be simulated in cloud-based test plans if this is how additional capacity is pressed into service when needed.
- Distributed cloud testing: Cloud-based software applications could be used from geographically distributed locations. This geographic distribution, and consequently, a variety of ISPs providing the service, needs to be simulated for realistic testing. There are commercial distributed test simulation services that have servers around the globe on various ISPs that can simulate user inputs from these locations. Cloud test plans could include the use of these services to increase the quality of the test plans.
- Multi-tenancy testing: Cloud-based software applications could have multi-tenancy that is subject to separation of clients at various levels and combinations – separate Web servers, separate database servers or data separation at the database level on the same server. Test plans may need to be designed to account for these and design in appropriate tests for these.
- Graceful degradation testing: Cloud-based software applications should be designed to degrade gracefully in performance instead of the servers just crashing in case of excessive numbers of users logging in and using the application at the same time. It is always possible to design the cloud application to restrict the number of logins beyond a certain number so that reasonable performance is guaranteed for those that are allowed in. Graceful degradation testing is an important component of cloud testing plans.
- Connected/disconnected operations testing: Cloud-based software applications could be designed to work only in connected mode or when an internet network connection to the servers is available. Nothing is stored on the client side, and the server side of the application stores all the data. Disconnected modes allow cloud-based software applications to work either in connected or disconnected modes. When working in disconnected mode, data is stored locally and when an Internet connection to the server is available, the locally stored data is synchronized with the server and vice versa. If this functionality is designed into the cloud-based application, the test plans need to reflect it.
- Cloud portability testing: When private clouds are mixed with public clouds, portability of the cloud-based application cannot be taken for granted. Test plans should include testing the high level functionality to ensure Cloud portability. Sometimes even in private clouds, if the system settings on the Web servers or database servers are not identical, bizarre application behavior could be encountered. If that’s the case with private clouds with servers that are completely under your control, public clouds need to be tested thoroughly, just to make sure.
Cloud-based software applications have some additional characteristics compared to non-cloud-based ones. These pose additional challenges but with a systematic, comprehensive approach to test planning, these could be handled appropriately.
About the author: Nari Kannan is currently the Chief Executive Officer of appsparq Inc., a Louisville, Kentucky-based mobile applications consulting company. Nari has over 20 years of experience in information technology and started out as a senior software engineer at Digital Equipment Corp. He has since served variously as vice president of engineering or CTO of six Silicon Valley startup companies, working in the areas of business process improvement, IT consulting, automotive claims processing, human resources and logistics applications. He can be reached at firstname.lastname@example.org.
This was first published in November 2011