In November 2009 Marisa Seal and Chris McMahon went to work for a company called 42lines. Among other duties, they were responsible for starting a User Interface (UI)-level test automation
Requires Free Membership to View
project from scratch. After a comprehensive review of the state of the practice for
driving browsers and for managing large numbers of automated UI tests, they
decided to build a bridge between the great browser driver Selenium and the great test management
wiki Fitnesse. The project, now called Selenesse, was released to the public in February 2010 as
open source under the GPL license. In this article, they describe the process by which they decided
to create a new bridge between Fitnesse and Selenium that they call Selenesse. Their experience is
immensely helpful for anyone considering a test automation framework for driving web browsers.
Chris:
I have done a lot of UI test automation in the last three years, and for 42lines I had a very
specific sort of test framework in mind. I knew we would use either Watir or Selenium to drive
browsers, and I wanted to be able to maintain the test suite in a wiki of some sort. At a company
called Socialtext (see Matt Heusser's chapter in the book Beautiful Testing "Peeling the Glass Onion at
Socialtext"), we scaled a UI test suite from 400 individual test assertions to more than 10,000
over a 2-year span, and being able to manage such a large suite in a wiki was critical to the
success of the project.
Anyone wanting a wiki for test management purposes clearly must consider Fitnesse for the purpose. The Fitnesse project is nearly ten years old and has a vibrant and active community behind it led by "Uncle" Bob Martin. I was hoping to find a robust way to use Fitnesse with Ruby and Watir, but the Fitnesse ecosystem outside of Java and .NET is not very rich. 42lines is a Java shop; the main Fitnesse development happens in Java; the main Selenium API is implemented in Java; therefore I somewhat reluctantly abandoned Watir and chose Selenium as the browser driver for Fitnesse.
With a wiki (Fitnesse) and a browser driver (Selenium) in hand, I started to research existing projects that bridged the two. I narrowed my choices to three projects: Fitnium, StoryTestIQ, and WebTest. In short order I eliminated Fitnium and StoryTestIQ as being unsuitable for 42lines, leaving WebTest as the main candidate.
 |
||||
|
|
|||
|
||||
WebTest is a project from the well-known software craftsman Gojko Adzic. Unfortunately, Gojko had abandoned the WebTest project somewhere around 2008, leaving WebTest only partially supporting the Selenium API, and not supporting at all any of the brilliant new features in Fitnesse like the Simple List Invocation Method (SLIM). However, I was able to swap out the ancient versions of Fitnesse and Selenium that ship with WebTest for up-to-the-minute versions, and WebTest still worked. This speaks to me of good design.
This is where luck came in: my colleague Marisa Seal started at 42lines the same day as me. She has worked extensively with DbFit, the database connection engine for Fitnesse. She and Gojko presented a talk on DbFit at the Agile2008 conference. Marisa agreed that a Fitnesse-Selenium bridge based on WebTest was a good idea, Gojko made us administrators, and we began building what would become Selenesse.
My role on the Selenesse project is mainly to be the Product Owner. I have a clear idea of how I want Selenesse to behave. Marisa, who is a far better developer than me, has written all the Selenesse code.
Marisa:
|
|
|
|
|
|
Once Chris and I began developing Selenesse and using it to
automate browser-driving tests, three Selenesse requirements evolved within a matter of
weeks:
- Compatible with Fitnesse SLIM test engine
- Compatible with Selenium-IDE test recorder/analysis tool
- Each interaction with a page element is an assertion
Although I've used Fitnesse with the FIT test runner pretty extensively over the last four years, I wanted to be able to take advantage of features available in the SLIM test runner, especially "scenario tables". Our initial version of Selenesse was developed for use with the SLIM test runner, but a limitation in SLIM required us to implement wrapper methods for many of the Selenium RC methods. The resulting side effect was that users familiar with the Selenium RC API would have to learn new names for frequently-used methods such as click and type.
Initially, Chris and I decided to follow a naming convention for new methods that were developed to work around the SLIM limitation, but that proved to be cumbersome both from the perspective of maintaining the Selenesse code base and when using Selenesse to write tests. Thus, our second requirement for Selenesse was born: Do not change the name of Selenium methods if possible. However, we could not adhere to this rule due to the SLIM limitation. I contacted Bob Martin and suggested a new feature for SLIM that would enable us to keep a majority of Selenesse method names identical to the corresponding Selenium RC method names. Bob Martin welcomed the contribution and the 20100103 release of Fitnesse included the new SLIM feature.
The third requirement was suggested by Chris: Treat each interaction with a page element as an assertion. Why? Acting upon a page element implicitly asserts that the element exists. What this means from an implementation perspective is that methods such as click and select return Boolean values indicating whether the acted-upon page element was present. This is a difference between Selenesse and Selenium RC; the corresponding Selenium methods do not return Boolean values. An unintended benefit of this implementation is that the absence of an expected page element results in a failed assertion instead of an exception, which improves the readability of test results and analysis of test failures.
More important than how the requirements evolved is how the implementation of those requirements benefits Selenesse users. SLIM compatibility enables users to create and invoke scenario tables -- think of them as templates – so that common actions such as logging in to a website can be used in multiple tests without having to write each step of the action in each test.
Scenario table definition
!| scenario | Log in as | userName | with password | passWord |
| open | /login.html |
| type; | txtUsername | @userName |
| type; | txtPassword | @passWord |
| clickAndWait | btnLogin |
Scenario table invocation
!| script |
| Log in as | marisa | with password | TestingTesting123 |
Keeping Selenesse method names identical to the corresponding Selenium RC method names means that test steps recorded in Selenium IDE can be converted to Selenesse tests with very little tweaking. A wiki format converter is included with Selenesse, in which recorded actions such as open("/somePage.html") are converted to | open | /somePage.html |. Updates to the wiki format converter are currently underway, the goal being to eliminate the need for any tweaking when transferring IDE-recorded statements to Selenesse tests.
Treating actions upon page elements as assertions benefits Selenesse users in two ways:
- Assertion pass/fail data reported by Fitnesse includes page element action counts
- Actions that fail due to the absence of expected page elements result in failed assertions, which are easier to identify than exceptions when visually scanning test results in Fitnesse
An Invitation to Try Selenesse
Selenesse is available for download or checkout at here. Selenesse comes with detailed
installation instructions as well as an executable test of a Google search that illustrates the
most important aspects of the Selenesse API. A Selenesse mail list is available here. Marisa and Chris invite you to try
Selenesse yourself.
About the author: Chris McMahon is a software tester and former professional bass player. His
background in software testing is both deep and wide, having tested systems from mainframes to web
apps, from the deepest telecom layers and life-critical software to the frothiest eye candy. Chris
has been part of the greater public software testing community since about 2004, both writing about
the industry and contributing to open source projects like Watir, Selenium, and FreeBSD. His recent
work has been to start the process of prying software development from the cold, dead hands of
manufacturing and engineering into the warm light of artistic performance. A dedicated agile
telecommuter on distributed teams, Chris lives deep in the remote Four Corners area of the U.S.
Luckily, he has email: christopher.mcmahon@gmail.com.
This was first published in March 2010
Join the conversationComment
Share
Comments
Results
Contribute to the conversation