is with static, hard-coded data in tests. When the underlying system changes, the tests fail not because of any actual problem, but because the data they use to manipulate the system has become stale. This tip will help you understand and deal with such challenges.
For simplicity, the examples here are shown using Selenium commands and some FitNesse features, but the underlying principles may be applied to any browser driver in any sort of test framework.
The problem with AJAX is that while the browser reports to the test framework that the page has loaded completely, within the page itself, elements to be checked by the automated test do not yet exist.
The naive way to automate such testing is to use a pause() or sleep() command, such as
This approach has two drawbacks. For one thing, adding many pauses like this causes the whole suite to run much more slowly than necessary. For another thing, on those occasions when "elementid" takes longer than 10 seconds to appear, the test will fail even though the underlying application may be correct.
A more sophisticated approach to this problem is to use a waitFor statement. For example, waitForElementPresent causes Selenium to poll the page in question for a certain amount of time, until the requested elements manifests itself:
Any test suite of reasonable size will quickly accumulate a large number of waitForElementPresent/click combinations, so it makes sense to abstract those calls into something easier to write. When using a programming language, this operation would become a method in the framework code. FitNesse allows the user to abstract such things into modules called "scenarios" like so:
!| scenario | Wait for and click | elementId |
| waitForElementPresent | @elementId |
| click | @elementId |
Notice that this scenario takes an argument "elementId", so using the module in the test itself looks like:
|Wait for and click||myelementid|
Over hundreds of tests, this sort of abstraction saves many extraneous lines of typing and also makes the tests much more readable. Of course, more sophisticated scenarios/methods are possible. The conglomeration of such abstractions is often called a DSL, Domain Specific Language.
Checking sort order
One common problem for browser automation is checking the state of the page after having performed some action that should change that state. A common example of this is checking the order of sortable data after having clicked something in the page that should change that order, like a column header.
The canonical way to check particular bits of text is with regular expressions. But the syntax for regular expressions is fairly complex, and checking sort order is a fairly easy operation. Selenium and a number of other tools supply a simplified implementation of regular expressions called a "glob." The glob character to match any text is ‘*’.
Say for example there is text in the browser page ‘AAAAA’ followed somewhere else by ‘BBBBB.’ After clicking something that affects sort order, the page should show BBBBB then AAAAA. A test for sorting would look like:
|Wait for and click||sort_change_element|
Since the ‘*’ character matches anything, there can be any kind of stuff between the strings AAAAA and BBBBB and the test will pass correctly, but if the sort_change_element fails to sort the data on the page, the test will fail correctly also.
Avoiding static test data
In some systems, it may be difficult or impossible to control the state of the test data. Two examples of such systems are those where data is in a particular state at a particular time, and manipulating the time in the system is too expensive to undertake. Another common issue is a requirement to run a suite of tests in multiple environments, where the data in those environments differ from each other.
The solution to these sorts of issues is to extract correct data from the test system at run time, store that data in appropriate variables, and use those variables in the tests themselves.
There are several approaches to doing this. One common but naive approach is to have the test suite itself do a call to a database containing appropriate data for the tests. This approach is fragile, though, because if the database changes, then the test framework itself must be maintained to keep in step with the database changes.
A better approach is to have the test environment expose data for the test framework to consume. That way any changes to the database or the underlying system are handled as an automatic consequence of normal development activity. One very attractive way to expose such test data at runtime is via a REST endpoint.
REST is an approach to creating Web Application Programming Interfaces (APIs). REST stands for REpresentational State Transfer. To oversimplify, in a REST API, the client tells the server how the system should be, and the server either says "yes" or "no" in response. REST endpoints may expose data in any format, but XML is common. This is convenient for tools like Selenium that offer XPath locators for Web pages. For example, a REST endpoint may offer the following XML document:
<baz id="GHI" ></baz>
There is a bit of FitNesse and Selenium magic in the following example, but it should be clear that it is possible to load a variable at run time from an XML document exposed as a REST endpoint:
The value for $BAR is now "DEF" and the value for $FOO is now "ABC."
Abstraction is key
Test automation is programming, and a key aspect of good programming is the DRY principle. DRY stands for "Don't Repeat Yourself." If your tests have many waitFor/click statements, abstract them into a method or module so you only have to say, "Wait for and click" once when you need it. If you are managing multiple sets of hard-coded test data for multiple systems, instead have the framework talk to those systems to get the data needed by the tests. And of course, always check that sorting works.