With automation from check-in to build, is there still a need for manual testing?
My initial thought was a definitive, “Yes.” Then I thought again and thought, “Of course.” On the third go round in my head I thought, “Well, it kind of depends on the situation.”
I think the answer I am most comfortable with is the third one.
There are certainly situations where good automation can assist the project team and provide solid data points for the team to review. This can provide valuable insights into the behavior of the system, while speeding through mundane tasks needed so the team can review the results. In some cases, manual testing will tell the project team little more than evaluating the results of the good, solid automation will tell them.
Still, many of the implementations of automation I’ve seen in the wild consist of “happy-path” testing with little regard to variance processing. If the software is to be used by human beings, it strikes me that at some point there are instances where manual testing will tell the project team little more than evaluating the results of the automation will tell them.
There are also situations where manual testing is absolutely essential. Systems that require physical manipulation or intervention are one example.
One question I have myself revolves around the costs related to manual and automated testing and the information that can be gleaned from reviewing the results. I find myself considering the costs of maintaining the automation suite and framework for systems that are in a state of flux or have regular releases or where given aspects are changing faster than the automation can be changed.
The balance of course is the cost of manually creating, maintaining and executing tests that can be run in addition to the automated processes, or in place of them.
In the end, the context of your situation, the need to be addressed and the questions to be explored should make the determination on how much, if any, manual testing will be needed or appropriate for your environment.
This was first published in June 2012