Are you someone who doesn’t have automated tests yet and you’re wondering if it’s worth the investment? Or maybe you have automated regression tests that aren’t working for you? Do you feel squeezed for time to do exploratory testing? Do you find yourself skipping some tests just because doing the same keystrokes over and over is sapping your will to live? If you answered ‘yes’ to any of these questions, there is a solution. There are multiple ways test automation can pay off. In this tip, we’ll look at how automated regression tests can be used to enhance exploratory testing, and make our work more enjoyable.
The multiple purposes of automated test scripts
Modern technology multi-tasks. Mobile phones are a good example: with a typical Smartphone, you no longer need to carry around a camera, music player, game device, books or a laptop. It doesn’t just do those other functions, it does them well.
Automated test scripts also serve multiple purposes. We write tests up front so that we know what code to write and when it is finished. Creating the tests is a team activity which enhances collaboration and communication. Automated test scripts provide up-to-date documentation about how the system behaves. Automating our regression tests not only provides quick feedback as we check in code changes, keeping regression failures out of production, it frees time for all-important exploratory testing to find out whether the software delivers
Familiarize yourself with automation options and tools
We’re lucky these days to have such a wide range of test frameworks and drivers available, along with many scripting languages. Each team can experiment and find the tools that work for its unique situation.
Over time, invest in automated regression tests at every level. Test-driven design not only helps programmers ensure their code is well-designed, it produces a suite of automated regression tests that have a super return on investment. You can use tools to test at the API level, basically replacing the user interface. GUI test frameworks and drivers let you test through all the layers of the software, using it like a real end user. Performance test tools will allow you to simulate load and stress your system before deploying to production. Test suites at all levels can run allowing for continuous integration, giving continual feedback throughout each day.
GUI automated tests are extremely useful in saving you from tedious repetition. In A case study in the use of automation, I give very some very specific examples of how, through the use of GUI testing, our team was able to test every combination of available options on a Web application that manages 401(k) plans. Testing this manually would not have been feasible.
Similarly, automated tests can be designed to let you run huge amounts of data through your system. You may even be able to take data from legacy systems to pump through your tests to find out if certain types of data will cause errors to occur.
Browser compatibility testing is another test made easier, thanks to GUI automation. By having scripts that automatically go through each function, you can run those tests across supported browsers easily to ensure there are no oddities that occur.
Design your tests for maintainability, and refactor them often. Careful design makes them flexible. For example, use variables for test inputs and results instead of hard coding them, so you can easily test a different scenario. Extract duplicate code into modules, mixins or macros, keep the design as simple as possible, and have each test set up and tear down its own test data where possible. As a result, you’ll find you can use many of your test scripts and fixtures for exploratory testing.
If you already have automated regression tests, look at them with a fresh eye. Can you use them for more than one purpose? Maybe you can use existing tests to set up data and scenarios for exploratory testing. It may require some refactoring to make the tests runnable in the right context, and pass in runtime variables to get just the situation you want for starting the test.
If you’ve been putting off test automation for fear that it’s too difficult and costs too much, get your team together to think about the potential return on investment. Automating your manual regression tests will free up time for more in-depth exploratory testing -- you won’t be spending all your time finding bugs that could easily have been caught with a unit test or functional test. In addition, you can design your regression test scripts to make them useful for creating exploratory testing scenarios. There are many other compelling reasons to automate tests, but these alone make it worth the effort.
This was first published in February 2011