Thanks to a move toward automated functional testing, Arizona Federal is "well on the way to greatly improving service to members," said the credit union's vice president of information technology, Kevin Bingham. That's because the IT staff no longer has to devote a considerable amount of time to manual testing and instead can focus on providing product services to its members.
"Our main goal was to relieve front line staff of testing so they could serve members, which is their job," Bingham explained. "I had implemented automated testing at a prior role [in a different company] and knew it could be implemented here."
The Phoenix-based Arizona Federal, established in 1936, manages more than $1.6 billion in assets from 216,000 member accounts in 25 branch locations. The credit union does not do any in-house development, but rather gets all its software applications from third-party vendors. For its core software application, which manages member accounts, Arizona Federal receives quarterly release updates. Bingham's staff was responsible for coordinating all the manual testing with the different business units, which was "a big function of ours," he said.
Including his staff and staff from the various business units, it took "anywhere from between 20 to 40 individuals throughout the lifecycle of testing. It was a pretty involved process of coordinating everybody," Bingham said.
For example, he said, certain groups would need to test first, so they would download the release and the documentation, identify the things changing in the environment, identify the testing environment to install the release into, and put up documentation for the business units. Then there would be a kickoff meeting with all the business segments, discussing the schedule for testing, which Bingham said was usually 60 to 90 days, depending on the size of the release.
Then, he said, "individual business units were responsible for finding time during the day to perform the tests they'd documented, then they sent test scripts and documentation to my team and log any issues they identified. We would then work with the core vendor and products services team to fix the issues identified."
It was taking significant time and resources to go through this process, Bingham explained. "With quarterly releases, we were constantly going through this exercise."
Relief from automated testing
Bingham recommended to senior management that they implement automated testing. He had evaluated automated testing products in a previous job, and based on that evaluation he recommended Compuware's TestPartner for Arizona Federal.
TestPartner is an automated testing tool with a storyboard-based visual approach that lets application users capture test scenarios. Arizona Federal began implementing the solution last June. Rather than trying to put test scenarios in the software for all of its applications, Bingham said they decided to focus first on the business unit that required the most resources in terms of testing, and that was branch services.
A test analyst worked with the business partner to record the manual test scenarios and identify any gaps in the manual testing. The business users can sit down at the TestPartner console and record their steps, and the analyst analyzes them to enable them to run in different scenarios, Bingham explained. To date, there are 4,100 tests scenarios being tested with TestPartner for branch services.
The test functions the credit union has defined can be scheduled to run any time once a new release is installed in a directory, he said.
"If there's a problem, the software identifies it and the steps up to where the problem occurs," Bingham said. "It gives you screenshots so you can see where the problems are. Then we provide that information to the core vendor to fix the issue."
Bingham added, "Now, we're only utilizing one resource, the test analyst, executing the scripts; we're not having to pull in all the business units. With automation we can run through all the scenarios in three hours."
Of course, no automated testing solution completely eliminates the need for manual testing. For Arizona Federal, for example, Bingham said, "there are certain things you can't test [through TestPartner], like depositing money in an ATM or taking money out. The software can't get up and do that. There are still some things people have to do, but it's dramatically less than it used to be."
In addition to improving productivity, automated testing also helps with quality, Bingham said. For example, in running some manual and automated tests concurrently for branch services, Arizona Federal identified six issues they missed in manual testing. Also, if a problem is found or a field changes or moves to a different order, Bingham said the software allows you to accept the change without having to recode the script.
"So you have versioning," he said. "I think it's improving the quality of the testing, and identifying items a manual tester may have overlooked."
Bingham said because TestPartner is easy to use, the business users are excited about the automated testing and want to create more scenarios. "I thing it's a more thorough testing than manual because it involves them in the process," he said.
Bingham said it took only five months to get a return on investment for TestPartner, running one test station. His goal is to have a lab set up with TestPartner running concurrently for each business unit.