What things should I take care of while testing an application in different browser/operating system combinations, i.e., browser compatibility testing?
Cross-browser is a critical element in Web application testing. The first place to look would be to your customer or project manager to determine if there is a published list of supported browsers. If there is such a list, and it includes multiple browsers, the next step is to determine what percentage of browsers to cover. If your operations team keeps statistics, you can request a browser access report from the site itself -- there's no better information than a relatively fresh list out of the access logs.
If you can't get really accurate log information, and if your project is not limited to a list of supported browsers, your first stop should be W3Schools.com's browser statistics pages. From this list, you'll get a general idea of both the current ratios of browsers, as well as the trends for each. This will help you prioritize your testing.
With an idea of the browsers you want coverage on, the next step is to determine how much coverage. Generally, teams that I work on split browsers into two categories: tier 1 and tier 2. Tier 1 browsers (IE 7 and Firefox would be my tier 1 today) are given equal and full test coverage, whereas tier 2 browsers are given an abbreviated amount of coverage.
Before I prioritize test cases and such, I look at my overall engineering team -- developers and testers. For the most part, each engineer has a preference for which browser to use primarily in development, testing and regression. Sometimes I can achieve 50% coverage right there (although I notice there's a strong tendency for engineers to shun IE and stick with Firefox).
The next step I take is assign coverage. Generally, I'll ask my test team to split coverage so I'm getting equal numbers using IE and Firefox. I then ask each tester to rotate browsers, to ensure their features are adequately covered.
So with coverage questions answered, your next question is about regression testing. I'm going to change the question a bit, to be "Now that I have tier 1 browser coverage, how to I cover tier 2 browsers? What tests need to be run?" The answer here is a bit trickier, but let me take a stab at it. I generally give tier 2 about 10% of my time -- on a two-week project, I'll have my team spend a day running regression tests against tier 2 browsers. This means testers must time-box their testing, and they need to be particular about what tests they run.
A final point of focus is to look at any code that has special-cased certain browsers. You need to make sure the special-casing is applied at the appropriate time, to the correct browser.
That's a high-level overview -- take the principles here and apply them to your daily effort.
Dig Deeper on Topics Archive
Related Q&A from John Overbaugh
Learn what's behind AWS outages and how to fix failures before they happen. Continue Reading
Learn strategies for best security test strategies for SaaS cloud. Continue Reading
Expert John Overbaugh identifies the three top concerns of the test manager and offers advice on how to stay ahead of the curve when it comes to ... Continue Reading