WavebreakmediaMicro - Fotolia

Manage Learn to apply best practices and optimize your operations.

Seven ways to know when to automate testing

Everyone wants automation, but knowing when to automate testing is the trick. Expert John Scarpino gives you seven questions to ask.

The word automation leads us to think of efficiency, return on investment, ways to cut expenses and methods to bring high-quality products to market faster. That's how the test automation process sells itself to a buyer.

However, "automation" used in regard to software testing must be taken a little differently. Knowing when to automate testing is a very strategic process, and must be managed by an experienced testing team.

In business results are expected yesterday, and upper management is quick to point blame at the software testing team when a project is behind schedule. Why isn't this software/process automated? If it were automated, testing would go much more quickly. We must automate it as soon as possible.

If only it were so simple. Before invoking test automation, what's really needed in situations like these is more thorough planning, less wasted time, and more knowledgeable resources?

I've learned that last-minute calls for automation are akin to a Hail Mary play in football. Most of the resources and time for the project have already been exhausted and management is looking for a quick fix to get it back on schedule. By the time the project rolls around to the software testing team, it is already so far behind -- usually due to glitches in development and business requirements -- that no amount of automation can make up the time lost. In addition, there is a lack of understanding within upper management about how test automation works and the preparation required to set it up. To be sure, the software test manager makes no friends in management when he must tell them automation cannot be done when they want it done.

Automation is valuable if planned in advance and implemented correctly, but automating a test process creates value only for a functionality that requires minimal change. This includes the functions of a software application that has completed the development lifecycle, has been fully tested, and the results of which have been approved by the organization's subject matter experts. However, if these areas require any coding change, then automation must be reapplied.

As software test engineers, we must educate our managers -- and in some cases, our own peers -- about when to automate testing and how it should work. Here is a quick list of considerations about when to automate testing that you can present to management when a project begins, to ensure that everyone knows what to expect:

  1. Is the development code stable?
  2. Has the project team accepted the functionality of the application and fully signed off on it?
  3. Does the team fully understand the test automation tool, and judged its capability to be used successfully for test automation? Has the team completed a proof of concept test and what were the results?
  4. What is the development plan in place, to understand when development will be completed and when automation can begin? Beware inflexible deadlines, which can lead teams to take shortcuts or build poorly constructed automated tests.
  5. Who will write the automation scripts? If the testers who execute the manual tests are the same testers who create automation tests, will this negate the productivity of utilizing these resources?
  6. What is the maintenance plan for how to complete after-testing to focus on automation areas?
  7. How well does the project team know the software vendor, and the software testing industry?

Next Steps

Automated software testing basics

Manual versus automated software testing

Four reasons projects fail

Dig Deeper on Topics Archive

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

What do you say when managers just want to automate everything?
How about... so when do we automate away your job.... and then when they give the logical objection, and talk about the need for human control in decision making, tell them, the same applies to your testers.  
Oh, so many, oh so easy.... 

"So, where are you working next...?"

"Agreed. Let's start with your job."

"If I'm automated out of my job, can I get early retirement...?

"Great question, boss, but I'm gonna need lots a time to work on just the right solution. Let's see, uhm, er...."
Oh my gosh, I love these answers! I'll have to tuck them away in the back of my mind. Not that I would ever be brave enough to say these things, but I would have love to hear someone say this to one of our last couple of CIO's that had the "automate everything" attitude (they're both gone now). 
In my experience, people may have a need to fulfill the official request even though they understand the absurdity of it. Then some work is done and success is reported. It's a real challenge to break such illusion!
Once a client told me: we automated all tests, you just need to make them working. Heh.
If we automate everything what / who are you going to manage?
If they automated system goes down for some reason, can you fix it?

Call the automated help desk for support
1 for English, 2 for Spanish, 3 for Binary....
I am not sure I buy the 'everyone wants' argument at the start of the article.

However, there are two things I disagree with.

1. Test Automation is not Testing software via automation.  It is at minimal checking, and it will never have that emergent aha that's different moment, except to the things upon which it acts and asserts.  It does not replace testers or tester's time, if anything it adds more overhead to a project.
2. If one is going to do it, waiting for stability is too late to start.   Look there are places where checking via automated checks does help and is fairly useful, but should fall to the developers not testers to writee generally (I speak of unit checks).

I also feel it focuses on Commercial tools too much.  Automation does not require any special high cost tools.  You can use them of course, but buyer beware.