The Agile development movement does not necessarily endorse tools. You can be agile and employ no more technology than a command line interface, a unit tester and some index cards on which to write requirements. But tools have evolved in recent years to better support Agile efforts. Among these newly evolving tools are several directly pegged at supporting a new type of project management.
Some tool types, particularly, have become linked with Agile development. According to Carey Schwaber, senior analyst at Forrester Research, Agile-specific project management tools, continuous integration build tools, and automated testing tools have become closely associated with agile processes.
"Agile shops focus their investments on tools that the entire team will use, layering Agile project management tools on top of testing tools on top of build management tools on top of software configuration management tools," Schwaber writes in the Forrester Report entitled "The Truth About Agile Processes."
Agile teams manage their work differently, according to Schwaber, and as a result, they call for a different breed of project management tool. Still, some teams rely on spreadsheets and wikis to keep things on track. Schwaber and others note that these ad hoc techniques can be stressed as Agile becomes a mainstream practice employed by larger development groups that tackle bigger projects.
What is Agile?
Agile processes came on the heels of the Extreme Programming [XP] movement of the late 1990s. XP focused on unit testing, pair programming and simplified requirements gathering -- some might say that a large supply of caffeinated beverages were also part of the XP parcel.
XP was in part a reaction by developers to too many failed projects -- projects that delivered late or didn't work, or they worked but were never used because the team "built the wrong product." There is some consensus that growing use of the Unified Modeling Language (UML) and the Rational Unified Process (RUP) -- which seemed to some to necessitate too much upfront analysis and modeling -- also influenced the move to XP.
In a way, agile processes -- best exemplified by the Agile Manifesto of 2001 -- systematized XP, providing reference principles and values for Agile development that gave a less-caffeinated cast on developers' desire for change.
Basically, the Agile movement seeks to cut the length and increase the frequency of feature iterations. The team itself is empowered to set deadlines. Delivering working software is paramount, and the up-front modeling and requirements gathering phrases of development are deliberately kept lean. Moreover, requirements gathering does not shut down after the early project phases; requirements are accepted long past the start of the project.
As a result, unit testing gains greater importance in Agile development. This is so bugs can be quickly isolated when software units are rapidly integrated. Also, effective automatic build tools become part of the Agile tool set. That is because constant iterations mean constant fast-paced integrations. Empowering the team means opening the view on the project, so enhanced project management tools are proving useful in agile development.
Forrester's Schwaber calls the new breed of project tools "purpose-built Agile project management tools." Among the players she cites as addressing this space are Rally Software Development, VersionOne and ThoughtWorks Studios. She notes as well that Agile templates are available, and that a Scrum template from Conchango has become a notable add-on to Microsoft's Visual Studio Team System package.
Recent project management product news
In 2007, Agile project management tools vendors were busy producing new product iterations with rapidity to rival their Agile customers. They are supporting new tool plug-ins, enhancing requirements handling, and so on:
- Rally's Release 2007.7 supports filtering on user stories and expanded filter criteria,
improved release burn-down charts, new notification rules, plus connectors for Eclipse and
- With v.2.6, TargetProcess added inline editing in lists, new iteration planning capabilities,
and Visual Source Safe Integration.
- A recent release of VersionOne's V1 reports open-source integrations with third-party tools Subversion and FitNesse.
For its part, well-known IT consultancy Thoughtworks moved into the tools business in a major way. In October, it released Mingle 1.1 for Agile software development teams. Coming just three months after Mingle's initial release, Mingle 1.1 is said to better enable project teams to schedule, track, prioritize and collaborate on tasks. Date properties in Mingle 1.1 help track and prioritize requirements, bugs and tasks. Too, the release expands the scope of filters for viewing story cards. Remote Subversion repository support was also added.
IBM's Rational tools group has sometimes born the brunt of Agile adherents' criticism, especially for its Rational Unified Process. It was widely viewed as a top-down process, somewhat oppressive for bench-level developers. But the company has never accepted that criticism totally. In 2006, Agile evangelist Scott Ambler joined the company to spread the Agile method. And, in 2007, IBM showed the first fruits of its efforts to create the Jazz development platform.
Erich Gamma, one of the most influential advocates of Design Patterns and now an IBM Distinguished Engineer working on Jazz, said Jazz follows up IBM's successful Eclipse effort. Ultimately, the new software is intended to support customized processes. "In developing Jazz, we've seen that different teams have different processes -- different ways, say, of handling change rules," Gamma said.
Jazz is still a (limited) beta undertaking, but the company showed its IBM Rational Team Concert, which is a Jazz-based collaborative portal designed to improve team productivity.
How Data Transfer Solutions does Agile
Processes change when Agile is in play. And tooling may change, too. Data Transfer Solutions development team members Chris Spagnuolo and Dave Bouwman worked informally with Agile methods for a year and a half before formalizing the adoption of Agile within their group. Their approaches are quickly evolving.
Prior to that, development involved long lists of requirements and very considerable upfront design for Spagnuolo, senior project manager, and Bouwman, lead GIS software architect. But with the advent of Agile, that has changed. Their team has opted for a Scrum-based Agile practice that streamlines initial requirements gathering, admits requirements additions through the life of the project, and centers on weekly builds of working software.
Early on, the team depended on an ad hoc mix of simple tools for project management. That has given way to a hosted Rally Software Development project management offering. Bouwman's and Spagnuolo's development practice at Data Transfer Solutions is centered on building spatial and related asset management applications.
"Before, we had tons of upfront requirements," Spanguolo said. "When we switched to 'Agile,' we did things differently. Now we gather requirements during the whole project, not just up-front. Clients have the capability to reprioritize weekly."
"With Agile we found you ship the [valuable elements first]. We quickly saw the benefit of putting functioning software in front of the client. Their understanding of the requirements changed," Bouwman added. "As soon as you show them something, you are going to get feedback on that. It is two-way feedback. They tell you if the increment is correct, and they get an understanding of how things work."
One thing that Agile has meant to Spagnuolo is that the users of his software can be agile as well. "They could think as we presented software to them, and be more flexible with their requirements," he said.
"Software is kind of an ethereal thing," mused Bouwman. With rapid iterations, and constant 'stakeholder' feedback, "users get a much more concrete feel. They will tell you the next step."
"When you do up-front requirements, it doesn't work that way. Just-in-time requirements gathering is a huge benefit," Bouwman added.
Spanguolo echoes an often heard comment -- that index cards and Post-its sometimes do the trick. As the number and complexity of projects expands, simple tools may not be enough. "We started out light on tooling, especially for requirements," Spanguolo said. He said his team used one spreadsheet to track user stories, and another to decompose tasks within iterations related to those user stories.
"That was working for us at the outset, but as we scaled up our Agile adoption and worked on larger projects among more developers, we outgrew the value of the spreadsheets," he said. "Managing the spreadsheets was getting to be pretty unruly -- about four to six hours per week."
Spanguolo's team also tried task boards to organize the work. As with the labor-intensive spreadsheet, low-tech task cards can consume effort. Said Bouwman: "Collecting task cards can become a big manual process."
Later, with the Rally project management software, the team was able to "track 'actuals,' roll results up for capacity planning, and generally ease things for the developers," according to Spanguolo. A reporting dashboard, he continued, lets developers efficiently line up tasks. Successful builds show up on the dashboard as green; if builds are broken, they show up as red.
Rally's tool is central, but it is one of many. Spagnuolo's and Bouwman's is a .NET shop. They use MS Build for the actual build and the CruiseControl.NET continuous integrations tool. CruiseControl.NET watches the source control tool, in this case, Subversion. When it sees a change has arrived, it kicks off a build process. Rally modules link to the build process or to the continuous integration engine.
It seems that fairly formal processes are emerging that are somewhat based around build management and workflow, powered by well-populated repositories and tools that provide intelligent views of project data. Nevertheless, each piece of what Burton Group Analyst Joe Niski has described as "the SDLC Infrastructure" contributes to the effectiveness of development projects and teams.
The key thing about the repository, said Niski, is that "it stores meta data about specific projects, where the source code control repository is for this project that we need to build." The repository, which is the essential data that the project management software relies on, also stores build results.
One could add that the requirements that are at heart of the Agile process are user stories. These user stories seem to be replacing use cases as a popular requirements gathering method. And, in many Agile efforts, multi-page user stories have given way to brief note card entries.
The project management tools emerging for Agile projects are now often able to store such note card entries as bitmap or jpeg files. This type of "storage agility" can be expected to expand in the future.