Problem solve Get help with specific problems with your technologies, process and projects.

Making sense of test automation: When and what to automate

Organizations want to take advantage of what automation has to offer, yet it needs to be cost-effective and time-efficient. Read expert Pete Walen's take on a commonly heard recommendation: "Automate everything that makes sense to automate."

I sometimes hear experts and speakers at conferences offering opinions that do not agree about test automation. I heard one recently say, “Automate everything that makes sense to automate.” I have some doubts about what he meant. Can you explain that?

Well, honestly, I’ve heard several people say things similar to that. I’ve said it myself a couple of times recently. The fact is, I cannot explain what others mean when they say that. I can, however, try and explain what I mean when I say that. 

The costs and complexity around automation will vary from one shop to another. The tools in use and the environment the System Under Test (SUT) runs in are only part of the question. The nature of the SUT will obviously play a role in this question. However, a small testing shop may be less able to absorb an aggressive automation development and maintenance effort than a large one. Even if a single tester is designated the “automation guru” and keeper of the framework and expert developer of scripts, if that tester represents 20% to 30% of the staff, it can be a significant drain on overall testing.

Even if the other testers are responsible for writing their own scripts to integrate with the framework, they will certainly need some help from the resident expert. This can reduce the overall testing output of the overall team. 

If the SUT has frequent changes to the UI, attempting to automate tests of the GUI can be problematic. Minor changes to a screen can usually be handled in stride, no? However, if there are monthly or even quarterly updates to screens, it strikes me that you would want to examine the cost of maintaining the automated scripts to adapt to these changes. 

Now, I know I’ve listed a number of considerations why you may want to rethink some of the ideas about the instruction to “automate all regression tests (or whatever) to reduce costs.” The right shop can take huge steps toward that goal. Some shops can, and have automated portions of their test suite successfully. However, none went straight in without careful research and planning. In that planning, you will be able to consider just what it is that makes sense to be automated in your shop.     

That is what I mean when I say, “Automate everything that makes sense to automate.”

Dig Deeper on Topics Archive

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.