Development organizations are gaining more experience with agile methodology, typically focusing their efforts on Scrum or Scrum mixed with other approaches, including waterfall, and are reaping benefits of process improvement and faster delivery times, according to the results of a recent SearchSoftwareQuality.com reader survey. They're also using agile more with distributed teams. Yet they still cite age-old problems like documentation and requirements, as well as resistance to change. And even as many organizations are just transitioning to agile, some industry observers say the next-generation of agile methodology is already emerging.
Asked what development process their organization employs, 56% percent report using agile processes, up from 45 percent from last year's survey. Test-driven development (TDD) also increased, from 19 percent in 2008 to 21 percent this year. Waterfall saw a slight increase, from 44 percent to 48 percent, while use of RUP dropped from 15 percent to 10 percent, and UML went from 14 percent to 10 percent.
Within IT, the results are similar. Fifty-two percent report using agile processes, an increase over 39 percent last year; and the use of waterfall remained level year-over-year at 36 percent. TDD also saw a slight increase, from 18 to 21 percent, while use of both RUP and UML declined slightly.
Although agile is a relatively new for many of the respondents, organizations are gaining experience and report fewer "newbies" than last year. In 2008, for example, 66 percent of respondents reported using agile for a year or less, and that number dropped to 46 percent this year. Those using agile for two-plus years rose from 18 to 29 percent, and those using agile three or more years rose from 15 to 24 percent.
This experience, though, is likely superficial, according to James Shore, consultant, author signatory number 10 to the Agile Manifesto, in commenting on what respondents cite as their top agile challenges: documentation, communication and resistance to change. These are the same top three challenges as the previous year, although communication was first last year.
"It's a reflection of the immaturity of agile adoption," Shore said. "Agile teams emphasize high-bandwidth communication, usually face to face or realtime vs. written documentation. If documentation is a bigger concern, then they're not doing communication well. I think it's human nature to take a new idea, adopt what's familiar and ignore the hard stuff. Adopting a two-week sprint and a daily standup [solely] is a superficial approach."
JP Steffen -- a survey respondent and IT project management contractor for Wells Fargo in the imaging services group -- said that while his organization is incorporating some agile practices, documentation "is a sacred process. Since it's multi-departmental in how projects are championed and requirements gathered, our documentation phase of our projects is pretty well entrenched. Reliance on documentation will be a hard nut to crack if they move to a more agile style."
Joel Cranford, a survey respondent, is business analyst at Nashville-based Healthstream, a provider of learning and research solutions for the healthcare industry that has been transitioning to agile over the past year. He described their approach to documentation as "just in time."
Forrester Research analyst Dave West said that while initial agile projects focus on building software rather than writing documentation, over time organizations are finding that some type of documentation is required to maintain the software over its lifetime. How teams document may change, though. "A wiki seems to be as valid as a Word document," he said.
Overall, there are still many projects that are large, expensive, lengthy and have a lot of components, said Theresa Lanowitz analyst and founder of voke inc. "Documentation is still very much required; especially with organizations where there is a lot oversight, like pharmaceuticals and financials," she said.
In terms of tools essential to agile development, this year's respondents cite, in order: bug tracking, requirements management, project management and functional test. In 2008 the order was: requirements management, bug tracking, project management and unit test.
Again, Shore sees those results as "a reflection of the state of agile adoption. A lot people are adopting agile without fully understanding it. One should work directly with customers or customer proxies and operate as living requirements, so you shouldn't need requirements management. And you should be using agile engineering practices that reduce defects; so with low defects you shouldn't need bug tracking as the highest tool on the list. I expect if you were doing agile to the fullest you would put unit testing on top of the list."
On the other hand, said Forrester's West, "I see a lot of organizations initially use bug tracking tools, but over time it evolves more to ALM [application lifecycle management] tooling like Rally or VersionOne." Particularly when agile teams are truly cross-department, products like "[Atlassian's] Jira and [IBM/Rational] ClearQuest become less usable" to those who aren't developers, he said.
Among benefit to agile, a majority of respondents (80 percent) cite an improvement in process, an increase from 68 percent last year. Other benefits are faster time to market (67 percent), about equal with last year; and fewer bugs (42 percent). Interestingly, 28 percent report lower development costs vs. 34 percent last year.
"One of the motivating forces behind the adoption of agile practices is to improve the software development process," said Kirk Knoernschild, an analyst with the Burton Group. "Beyond that, the goal in adopting agile is not solely to improve the process, but improve productivity, quality, speed of delivery, and establish a more collaborative relationship with stakeholders and customers."
However, according to Shore, that process improvement may be short-lived. "In the short term [some agile adoption] does make a big difference in the perception of control, morale goes up, and there's more transparency. But after a couple of years you get overwhelmed by engineering problems [with a superficial adoption]."
In terms of team make-up, agile seems to be slightly more popular with geographically distributed teams (55 percent), up from 48 percent last year.
"We're seeing the reality of modern working practices affect agile," West said. In an ideal world you'd be co-located, which is why we're seeing more use of tools and technology like Skype, IM, etc., becoming more relevant. And," he added, "the Facebook culture is more accepting of geographical distribution."
Knoernschild said distributed teams can "absolutely" make agile work, but he stressed the importance of having an infrastructure in place. "When you're sitting next to two other developers on your team you don't need much infrastructure, but when you're 2,000 miles away and separated by time zones it's more important to have an infrastructure that facilitates collaboration on many levels."
According to 59 percent of survey respondents, typical agile team size is 4-9 members (up from 54 percent last year), followed by 10-20 members at 23 percent, also up from 20 percent last year. There was a drop in the number of teams with 3 or fewer members, from 23 percent in 2008 to 8 percent this year.
"I don't know if I would necessarily say there's a right size for a team," Knoernschild said. "It depends on the size and scope of the project. As the team grows the formality of practices and the importance of infrastructure becomes more pronounced."
For those organizations that have not transitioned completely to an agile methodology, 43 percent said they were using both agile and waterfall. As to why, 24 percent said that the staff lacked an agile skill set. Other reasons include veteran staff with waterfall skills (18), and waterfall is working so why change (14).
That's the case for Steffen's group, he said, where waterfall is a very collaborative process. "It's not like some places with waterfall that have an initial meeting with stakeholders, draft the requirements and go away. Our PMs realized the weaknesses in that methodology, so they continue to have a role and almost daily conversations with the development team. So the emphasis on a collaborative environment between stakeholders and the development team is something we've jury-rigged into our waterfall process."
For teams that do utilize agile, Scrum is the predominant methodology. Just over half of agile teams use Scrum (vs. 40 percent last year), with declines in Extreme Programming (XP), Crystal and DSDM. "Scrum is by far most the popular project management methodology," Knoernschild said. "Typically you see Scrum applied at the project management layer, then organizations are scaling up by using a scrum of scrums, and practicing some of the XP practices at the development layer like unit testing, continuous integration and paired programming."
Healthstream is using a combination of Scrum and waterfall, Cranford said. "I wouldn't call it a 'mature' agile process; we've been using Scrum for the past year, and we're still shaking out the cobwebs," he said. "We're identifying what form of agile fits us—it's an amalgamation of mainstream Scrum and some vestiges of waterfall, but we're getting there."
"Scrum plus waterfall is the de facto a lot of people are using," Shore said. "That works for a couple of years, then they start to have problems with software quality and design quality."
The best mix, according to both Shore and Forrester's West, is a Scrum/XP hybrid. However, West said, there are other good engineering practices taken from more traditional approaches. "We're seeing less allegiance to one method and more acceptance to pick the best things, but still put that in some framework," West said.
But even as organizations work to define their agile efforts, Knoernschild said newer methodologies are emerging, like Kanban. "I wouldn't say it's an alternative to agile, but for lack of another word to describe it, it's the next generation of agile methods. It's an aspect of lean thinking. That's really where the next generation of agile methodology and software process improvements will be centered around."