BACKGROUND IMAGE: Zffoto/iStock

E-Handbook:

Agile automation techniques answer the need for development speed

Manage Learn to apply best practices and optimize your operations.

Automated testing in Agile is still much harder than you think

Automation is a key feature of Agile and DevOps, but it remains elusive. A recent survey from LogiGear shows companies are making very slow progress on both automation and Agile.

Automated testing in Agile -- and DevOps -- is as important, and elusive, as the Holy Grail. And with good reason:...

No development shop worth its salt can actually claim to be on the Agile and DevOps path without a healthy dose of automation. But that doesn't mean it's easy. And after all these years of Agile, automation still proves to be quite challenging.

Survey reveals test automation challenges

In 2008, software testing tool provider LogiGear did a survey of Agile practitioners to capture the "state of the union," and the company is currently doing a similar ongoing survey, this time in four parts. Dishearteningly, the results of part one show little to no progress when it comes to automated testing in Agile, according to LogiGear senior vice president Michael Hackett. Ninety percent of testers who responded to the survey said 50% or less of their tests are automated, 27% said just 10% or less of their testing is automated and only 5% said up to 90% of their testing is fully automated; 20% of respondents do all their testing manually.

These numbers don't surprise Steve Elliott, founder and CEO of Agile platform maker AgileCraft. When Elliott visits customers, he might see a lot of sophisticated, high-end automation or absolutely none at all. "It's one of those things that's really all over the map," he said. "Companies just really haven't figured out the automation in Agile piece yet."

But there's no question they need to -- and quickly. "There's even more pressure to be faster than ever before," added Forrester Research analyst Robert Stroud. "The velocity is just going to continue to increase and with it the demands on developers. Things are going to go as fast as the business can let them go." The answer to that -- continuous development/continuous integration (CI/CD) -- can really only happen if automation is in place.

Understanding the Agile testing process

But clearly that's easier said than done in the Agile and DevOps world. Automated testing in Agile has to begin with a complete understanding of the entire development process from start to finish. Companies can't just automate randomly; the process has to work seamlessly. When LogiGear did its survey 10 years ago, the biggest challenge with automation was management's lack of understanding the process, Hackett explained. And unfortunately, the exact same thing is still true today.

"There is a lot of misunderstanding about automation," he said. "So many people are completely unprepared, and if you don't have things already well along the path of significant automation in Agile, you are just going to get steamrolled. The real surprise to me about this survey is how much is still not understood -- like requirements creep and vague or bad user stories. Testers have been complaining about the same things for the last 20 years."

The survey also asked if the move to Agile -- and now DevOps -- has improved the development process. Again, the answers remained little changed over the last 10 years. Back then, about 20% of respondents said moving to Agile was an improvement, and that percentage is the same today.

"There are still a substantial number of people who think that things just aren't getting better," Hackett lamented. "The last two companies I visited had zero automation. It's not going to be good when companies finally wake up and are interested in modernization and digital transformation. It's going to be hard." 

Next Steps

What you need to know about Agile at scale

Is Agile still relevant?

Why Agile and DevOps remain a work in progress

This was last published in June 2017

PRO+

Content

Find more PRO+ content and other member only offers, here.

Join the conversation

4 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

What has been the biggest challenge when trying to automate in your Agile environment?
Cancel

"Testers have been complaining about the same things for the last 20 years".  And I would say longer in regards to Requirements and the information that needs to be contained in them.  If the recipe doesn't have enough information in it to start then how do you expect to make and bake the cake properly?  Also, the misunderstanding about "automation" by management is still the same as it has been for the last 25 plus years.  They expect a Silver Bullet and some type of magic.  "It's Automation, Not Automagic!" has been my mantra and saying for years.  The key is making sure the process and related work tasks of automation are baked into the overall equation for building the software.  This includes "testability" features being baked in from the start by development.  But this requires that management understands this and supports it.  The process model won't matter (Agile, DevOps, Spiral/Iterative, etc.) and automation won't save it unless all groups involved understand and support the work to be done.  You can't slapdash on automation to a project.


Jim Hazen

Cancel
Being in the test automation space with Qsome, I find this observation on point: 

"The real surprise to me about this survey is how much is still not understood -- like requirements creep and vague or bad user stories. "

You can't automate with any degree of effectiveness or surety if you don't understand your user journeys or have no plan of what the automation should achieve for you. 
Cancel
Getting automation also done in same development sprint is a big concern. APIs/UI are generally not available until 4th or 5th day of a 10 day sprint, leaving just 5 day to develop automation scripts and integrate in CI/CD pipe. Practically automation spills over to next iteration. Another challenge is the priority given for automation. When it comes to movement of a new feature to DONE, sprint teams tend to bypass completion of automation.
Cancel

-ADS BY GOOGLE

SearchMicroservices

TheServerSide.com

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

DevOpsAgenda

Close