Unimpressed with his first experience with Agile development, FamilySearch's senior quality assurance engineer Jeff Porter pushed for more training and helped put Agile practices on the right track. Still dissatisfied with Agile's ad hoc nature, he worked with the development team to mix Agile, Scrum and continuous delivery practices. The resulting methodology mashup has resulted in a decade of structured development and on-time delivery at FamilySearch, an online nonprofit genealogy services provider.
After over a decade of working with Agile at FamilySearch, Porter is an enthusiastic Agile practitioner and a regular speaker on Agile best practices at conferences, including the Fall 2013 STPCon. This story describes Porter's work as an Agile development change agent and QA leader.
Porter's road to Agile evangelism was, as mentioned, a bit rocky. At first, he saw developers interpret Agile as "doing inspect and adapt all the time" and never stopping to analyze or consider how the system will evolve over years. Developers, in his view, thought Agile freed them from doing analytical work, because they could just change their minds about project direction at any time.
To build structure into Agile methods at FamilySearch, Porter worked with the development team on a process that included Agile, Scrum and continuous delivery methods to define documentation, measure and optimize cycle time and encourage frequent commits, or checking work into source control. Operations and business staff were brought into development team. Finally, testing was done at every step, so that finished components of applications were ready to be put into production.
When a feature is 'done'
Defining done was a key addition to development practices, Porter said. Done means that the code is in place and has passed unit and all other tests; user and support documentation is completed; and special capabilities, such internationalization, are completed.
"Getting the team to define done and willingly adhere to it is the biggest bang for the buck of all processes we've added," Porter said.
FamilySearch's DevOps team uses Scrum's point system -- a ratings method that used an agreed-upon metric, like ratings of 1-10 for sprint estimation and review. First, a story, or business need, is determined and set forth as an application feature. Then that story and others are put into a short-length development project, or sprint. "In the past, at the end of the, say, two-week sprint, often it was unclear whether the story was complete and ready to sell off." The point system helps the team determine which stories are done and ready to be deployed and which are not.
The uncompleted stories are put into the next sprint, often alongside new stories. "If we get three days into a two-week sprint, and a story from the previous sprint is ready, then we put it into production immediately," said Porter. "This is just one way Agile methodologies enable continuous delivery for us."
As FamilySearch's senior QA engineer, Porter encourages developers to write unit tests to test their own specific code, but their tests get reviewed by QA. His team interacts with developers at every step in a sprint, so developers know what tests will be running and when. "I'm in the trenches with them, running tests ad hoc, trying to get a feel for what's working along the way," Porter said. The result is fewer failures when products get to final QA.
QA as the user's voice
The bulk of QA's work is functional and user interface testing. "We're the voice of the user," Porter said. "We work with the product owner and development team to make sure the design is user friendly and the functionality fits users' needs."
Being the "voice of the user" has become much more difficult during Porter's tenure at FamilySearch. "When I first started out here, our charge was to take a Web system to the general public, making it something that Joe on the street could understand," Porter said. In the current age of the savvy user, expectations are high. "You develop a great system, and the savvy user says, 'Is this all it does? Well, it doesn't serve me coffee.'"
Current projects reflect the demands of savvy users. Porter is working with FamilySearch's photos and stories group, which creates applications that enable users to add photos, tag people in the photos and then link them to people in the geological database. A stories project allows customers to write stories about their family and upload them as PDFs.
The savvy user can be helpful and harmful to software projects; helpful in providing feedback, harmful in challenging security. Hacking attempts have increased in the last few years. For example, steps have been taken to ensure people couldn't place macros into input fields, aiming to damage the system. A key role for Porter's QA group is researching what hackers are doing.
On the other hand, Agile development calls for constant user feedback, and a benefit is that savvy users are eager and willing to offer it. FamilySearch sites feature a prominent feedback button and a support group dedicated to creating feedback reports for the product owner. The feedback helps developers define business needs and develop features.
There's less user dissatisfaction about systems changes today. "When savvy users see a feature that behaves a little differently than it did a week ago, they think it's cool," Porter said. "They're tailor-made for continuous delivery." Less savvy users, however, complain about continual changes. "We try for a balance between these two different groups, creating help features for both of them."
Summing up, Porter said that balance was the key best practice in QA and Agile development. QA teams must balance user needs against development realities, and vice versa. Continual delivery must balance against the speed with which users can learn new features. As for Agile, one can believe in the overall theory of Agile, but must create a balance between which Agile practices do and don't meet a particular business' needs. "We found that balancing Agile, continuous delivery and other practices works better for our team," he concluded.
Which Agile practices does your team NOT use and why? The fourth one to respond will receive a copy of the book Discover to Deliver: Agile Product Planning and Analysis by Ellen Gottesdiener and Mary Gorman (U.S. only). Let us know what you think!
Six steps for disciplined DevOps software delivery