Automating software testing and development can shave time off QA processes, but automation should be a means to an end, not a goal in itself, according to Colin Dorman, director of technical support and software release management for Thomson Reuters.
"The goal of automating software testing is freeing up time for people to do work that adds value to the product," Dorman said. "Succeeding with automation calls for knowing when, why and how many technologies and processes to automate."
At Thomson Reuters, Dorman leads the second-line technical support and software quality-assurance (QA) teams for the company's Eikon financial analysis tool set. He also works with other Thomson product lines, including Thomson ONE, a financial content delivery service. He answered some questions for a SearchSoftwareQuality editor.
What is not an important goal of automating software testing?
Colin Dorman: I think people see test automation as a way of saving costs. I think that's a bad approach, particularly in terms of total cost of ownership. Automation is difficult to do well -- more difficult than people think it is. They usually just buy a tool -- that's a cost -- and spend a lot of time implementing it, [which is] another cost. Then, they have to spend time to actually create your sweeps. Then, there's the cost of maintaining the tool.
It's surprising, but the cost of automating software testing is often overlooked. No finance department I've come across has ever asked, 'What is the true cost of ownership for this tool? What is the maintenance overhead?'
What I've seen is a company spending six months to build this fantastic automation rig, which is then ignored. Six months later, it's defunct. It becomes an expensive waste of time.
Colin Dormandirector of technical support and software release management, Thomson Reuters
But there can be cost savings to automating software testing, right?
Dorman: Yes. If you can make people more efficient, you'll absolutely get more coverage with the people you have.
Do you see your peers in other companies rushing to automate test and QA tools?
Dorman: The opposite, I think. I see that people don't invest enough time in automation. No one actually has time to automate, because they are rushing through Agile sprints. They finish one and move on to the next sprint. What I see happening is a lot of technical debt that could have been prevented with automation.
On your test and QA teams, have you automated every tool and process you can automate?
Dorman: You would be kidding yourself if you said you were there. There is always something you can automate. I think the trick really is choosing your battles, and finding the right things to automate and looking for the biggest wins. Let's face it, some things just have to be tested by a human being. The problem with automation is it only does what you tell it to do.
What is automation's key value for your team?
Dorman: Automation gives you a lot of confidence that things are working. It empowers your QA testers to go off and do the most valuable tasks. We've been using TestPlant [eggPlant Functional] to automate routine tasks, for example. This frees up time for exploratory testing or destructive testing, where testers sit down and really pull the product apart, and try and do weird and wonderful things with it.
A lot of QA teams spend way too much time doing regression tests, and that's where automation really does help. Without automation, your scope of your testing is incomplete, because you're only going to do regression testing for what you can remember, which is probably three or four sprints back. And then, things drop off.
What is another area where automation isn't, but should be, used?
Dorman: Requirements is another place where development teams lose time. The product team comes up with requirements. Usually, the requirements work is put into some sort of a document, and that document is never carried straight through to QA. Do they drop it on the floor? I don't know.
I challenge my team to figure out how can we take the requirements that come from the product teams and turn them into test scripts. We developed a tool ourselves, a keyword-driven interface in something like an Excel spreadsheet. This sets up an extraction layer into our own API. So, we can just cut and paste or copy across from the [requirements] document.
How Agile can make software testing faster
The best tools for the testing job
Will automation be your new best friend?
Make the API testing process automatic