When Windows 7 becomes generally available in October, Ron Yun, director of quality assurance at Ellie Mae, wants to be ready. He's already got a head start on testing with the Windows 7 release candidate through the use of Skytap's cloud-based testing lab, which this week is announcing support for Windows 7.
A provider of software and services for the mortgage industry, Pleasanton, Calif.-based Ellie Mae Inc. needs to make sure its flagship Encompass mortgage automation system works on whatever platform or machine mortgage brokers happen to be using.
"We have no control of the platform or machine users are using," Yun said. "There are a lot of flavors of Windows and Web browsers, and you have to test and support the mainstream platforms. With Windows 7 the general release is October 22, and we expect some number of customers will be running it pretty much right away, so we better be ready and supporting it. We're doing some pretty heavy testing now."
Ellie Mae evaluated Skytap in May and started using the virtual testing lab in June. Yun said Ellie Mae itself does not have the Windows 7 release candidate in-house, and does not plan to upgrade immediately. Through browsing the Skytap library, Yun said he discovered Windows 7 "and in five minutes had it up."
While Ellie Mae has a subscription to MSDN, which would give the company access to the release candidate, having that availability in Skytap's library makes it easy to stay up to date, he said.
"The beauty is they keep their library up to date, so we can see what's new," Yun said. "Had [Skytap] not had that available we could've asked IT to build a Win7 system, but going through the IT department is somewhat painful. They're shorthanded, and I wouldn't expect results for a week or so."
The ability to quickly set up and tear down test systems was what led Ellie Mae to Skytap initially. Yun said the company has an in-house test lab, "but those systems are pretty much fixed with their version of operating system. We can change them, but it's not real easy. We were looking for an easier way to change platforms and combinations of different versions of products."
Creating a virtual system through the Skytap UI makes this task easy, he said, and then the testers can install the application and configure it they way they need to. "We can create a system to our needs quickly, use it, and when we're done we can keep it around or delete it. And it's all done in a cloud outside of our organization."
He added, "I think of a Skytap virtual system as a black box. I can say 'create a Win7 system for me' and it does. I can say 'add 2 gigabytes of memory' and it does. I don't know how it does it, but beauty is I don't need to know, and I get what I need."
Yun said he can do everything with a Skytap virtual machine that he can do on a local machine, as well as install any tools he's using on his local machine. For instance, Ellie Mae has been doing automated testing with HP's QuickTest Pro, which he said works seamlessly with Skytap.
One feature he finds especially helpful with Skytap, which he didn't have with the in-house lab, is the ability to take a snapshot of the system at any point in time, which makes pinpointing problems easier. "It's always a challenge to report to the developers and tell them how to reproduce a problem so they can fix it. Sometimes the configuration of the system when you see a problem creates the condition or the use case that triggers that problem."
With Skytap, he said, "now when we see a problem we press snapshot and continue testing. The developer can later log into that snapshot. It's a nice feature, and I can't do it on my local system. Previously we would have to stop testing and get a developer [to view the system when the problem occurred] or wait until a developer was available."
The ability to do early Windows 7 testing has provided a big productivity gain for Ellie Mae, Yun said, particularly for his team in Beijing, which does testing on both the desktop client and Web client versions of the Encompass application.
Previously, when Beijing needed to download a new build of the desktop version (typically about 500MB) from the ftp site, it would take almost three hours. "Now when they use a Skytap virtual machine, they're using a machine in Washington state [in Skytap's data center] and when they download to the Skytap VM it takes about two minutes—about the same time takes me to upload it," Yun said. "So we're gaining half a day of productivity on a new build. We do builds every day, so when they downloaded to the desktop [Beijing] was a day behind. With the VM they're right up to date, which is a big benefit."
The Beijing team has also seen productivity gains in testing on the application's web client. Ellie Mae hosts the Web-enabled version of Encompass in its Chicago data center, so previously the Beijing team would have the Web client on their desktops and communicate with the data center in Chicago, and response time was not great, Yun said. Now with Beijing using Skytap virtual machines, the communication is between Skytap's Washington state data center and Ellie Mae's Chicago data center. "The response time is much better, which makes them happier, and faster," he said
Going forward, Yun would like to see Skytap's reporting/auditing capabilities become easier to use. For example, his Beijing team creates and deletes the virtual test systems on their own, and Yun said he would like to be able to audit what they're doing more easily. "I can do it now, but I have to go to an administration area [of the system] and put in the parameters. I can get the information, but I think it could be easier."
To date, Yun said he's been pleasantly surprised with the results of the Windows 7 testing. "We've found some minor things, but nothing major."
In terms of being ready for when Windows 7 is generally available, Yun said "Skytap has given us a big head start."