Upper management's lack of support for Agile development is a common cause of software project failures. More insidious than abject failures are the undiscovered slowdowns caused by CEOs' and chief information officers' poor support of Agile projects, according to Agile software methodology experts. Agile success rides on management-driven organizational change, they say.
Misconceptions about what Agile is and how it works best often lie behind the absent-leader approach that is derailing or, at least, diminishing the success of agile software development projects, said Agile development veterans Bob Galen, Lisa Crispin, James Shore and Howard Deiner.
They spell out common leadership mistakes and suggest remedial measures for CEOs and Agile project managers in this article.
Agile and absent leaders don't mix
A common story about Agile adoption goes like this: A CEO reads or hears about Agile development benefits and, like Star Trek's Captain Picard, tells a development manager to "make it so."
That's not a myth, said Agile experts. First off, 67% of Agile initiatives were championed by executives, according to a 2012 Agile user survey by voke inc., a Portland, Ore. IT research firm. The survey also showed that upper-level business leaders were either unaware of or did not understand Agile development and did not participate in about 60% of projects.
For more than a decade, absentee management has been the blame for 75% of projects that consultant Bob Galen has seen. "In these cases, senior leadership doesn't realize that they have to play a part. They think they can just strap the methodology onto the technology team,” said Galen, a certified Scrum coach and author of the books Scrum Product Ownership -- Balancing Value from the Inside Out and Agile Reflections.
Agile tester and developer James Shore has been working hands-on in Agile teams since 1999. "In that time, I've seen a lot of failure cases and near-failures," he said. Shore was an Agile Manifesto co-signatory in 2001 and is co-author of The Art of Agile Development.
How CEOs' misconceptions hinder Agile
The importance of educating upper management about Agile is evident largely when their ignorance about the methodology messes up projects, said Galen. Here are management misfires he and his peers have seen frequently.
No. 1: Improving software development with Agile will take only a few months. "Management is sometimes surprised or unprepared for things to slow down as teams learn Agile," Shore said. It takes years, not months, to implement Agile development fully in an organization, Shore said, and experts agreed.
"Initially, they will be rewarded with at least small features delivered frequently," said Crispin. “After a couple of years, the team will be able to respond to business changes quickly and fully." Crispin is co-author of Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley, 2009) and Testing Extreme Programming.
Often, it takes as long as five years to realize the full benefits of Agile, Galen said. The good news is that the organization has been practicing and benefitting from continual improvement the whole time.
No. 2: Adopting Agile only requires bringing new tools to the development organization.
Often, a manager will buy an Agile project management tool that facilitates iterative development for analysis, design and some aspects of development and deployment. "And that's it. That's the only change made," said Agile and eXtreme programming practitioner and coach Howard Deiner -- and failure resulted in almost all cases.
No. 3: Agile development's primary goal is speed, to churn out applications quickly.
Some senior managers think Agile development is just a speed play, Galen said. "If we go at Agile we'll get three times more stuff." Crispin has seen management's focus on speeding up application releases without quality lead to heavy business losses. They must clearly understand technical debt -- the accumulation of application flaws if quality control does not occur -- in the call for speed.
"It's important to be able to deliver software that provides value frequently and at a sustainable pace," Crispin said. 'Sustainable pace' means that speed takes second place to good QA practices that keep technical debt low; practices such as test-driven development, regression test automation and exploratory testing.
No. 4: Only the development group has to change in Agile adoption.
"Management will say, 'We'll do Agile, but we'll keep everything the same,'" Deiner said. "They 'cargo-cult' and do 'Scrummerfall.'"
Agile brings with it organizational transformations. Agile requires change and continuous improvement, not just in development but also in all practices that touch products.
Deiner, Crispin and Gabel have seen that old-school command-and-control management styles usually do not play well with Agile. In addition, they need to become facilitative in their approach to lead people. Openness, honesty and transparency in communication are the logical places to start.
Upper management's paths to Agile success
Learn and adapt: Upper management can help their organization make a successful Agile transformation simply by being informed and genuine about the effort. Show, through actions, that they support the effort and expect there to be growing pains. Training is available from various sources, including Scrum Alliance. Beware, however, of thinking that a three-day Scrum Master Certification class will suffice. It's a good starting point, that's all.
Get involved: In all the successful Agile adoption Galen has seen, leadership actually led agility from the learning, understanding and implementation perspectives. "They would understand that it's not a technology play," he said.
Galen points to Salesforce's success with Agile. "If you look at successful Agile organizations, such as Salesforce, you can see that leadership embraced it and fostered it before even the organization did," said Bob Galen. Organizations' top leaders never relinquished their sponsorship downward into the organization. "They were always there."
Change cultural and management attitudes about success: Nurture a learning culture where failure is OK so that innovation can occur, said Crispin. Provide training and support for all team members in all roles. If this type of culture is in place, the whole organization will get better at delivering the right software at the right time. If a learning culture is in place, development and business teams will get better at delivering the right software at the right time.
Tips for project managers who need CEO support
Think Agile transformation, not Agile adoption. Use "transformation" during the initial and ongoing discussions with management. Here is how:
Ask for help. Ask for funds for training, for management to attend some meetings, for organizational change. Engage HR, operations, business and sales. "If the PM looks at Agile as an organizational development play, that thinking will spread," Galen said.
Introduce an Agile competency center to get in-house management and staff trained on all aspects of Agile. Then, move on to create an Agile Center of Excellence (CoE). Ask management to be on the CoE steering group. Acknowledge that you need steering – not just developer steering but organizational steering, Galen said. Very often, there's a nice complement among other CoEs in the organization, the Agile CoE and the Agile Project Management Office.
We'd love to hear about your own Agile failures – and successes! Post them publicly in the comments section below or send them privately via email.
Dig Deeper on Agile Software Development (Agile, Scrum, Extreme)