adam121 - Fotolia
For large organizations, the move to continuous development is easier said than done. The technical challenges are very real. Many aspects of testing and deployment must be automated to keep pace with continuous developers. That means writing, testing and managing code that can automate the building, testing and deploying of code. The result is what's referred to as infrastructure as code -- a fully automated development, test and production infrastructure.
Gil Clark is used to these challenges. Clark, development productivity architect for Intuit's small business group, has a fairly uncommon title. "In a smaller organization, I might be called a DevOps architect," he said. He explained that Intuit's small business group is responsible for all the business software Intuit provides for small businesses. Think QuickBooks and Turbo Tax for businesses. Clark's role is to make sure developers and testers have the best possible infrastructure in place to get their jobs done quickly and effectively.
Clark believes infrastructure as code may be the most important tool in his belt, and automation is a huge part of his strategy. Continuous development allows development teams to test every code commit and deliver working code as soon as it's ready. Keeping up with that pace would be impossible with manual testing.
Of course, some forms of testing -- like user acceptance and certain aspects of penetration testing -- still must be done manually. "But many Web services have no user interface. They're all back-end computing, and those can be 100% automated," Clark said. The bulk of Intuit's functional testing has been automated using tools like Selenium and Jenkins, as well as custom-built code designed specifically for Intuit's unique application architecture. "Every code commit gets automated unit tests, smoke tests and so on to catch any defects there might be," he said.
Currently, Clark said, only a few of his projects have fully achieved continuous deployment, where changes go into production without any manual intervention. Some projects have continuous delivery, meaning the code is pushed out to a staging environment where it is reviewed and approved before being sent into production. Other projects still lag a bit and require manual tests by quality engineers before they can be deployed. But Clark's goal is complete automation and full continuous deployment. "Infrastructure as code is the software that runs all that," Clark said. "It's like another name for continuous development."
Here's an example of what the build process might look like for a hypothetical Web service. "It starts with Jenkins -- which we use for a build server -- to automate the build process," Clark said. Jenkins would find or create a test server in AWS. "If the server doesn't already exist," Clark said, "we have to get the right Tomcat server set up with a certain version of Spring and the right operating system." Clark said he uses Chef to automate that build process and SaltStack to manage the configuration process. Then Jenkins would trigger Selenium to run all the automated tests to see if the Web service is ready. All of this is done with code, so Clark uses Perforce to manage the files that hold the code for each of those other tools, as well as the source code for the Web service under development.
Not everything can be developed in the cloud. Intuit maintains its own secure servers in its own development and test data centers. These hardware resources are not automated like AWS servers. Intuit has VMware in place on its on-premises hardware, but it's still fairly static, according to Clark. "Right now, we have to put in a request and then we get new servers a few days later." So for the future, Intuit is working on setting up an internal private cloud that could be automated the way its AWS infrastructure is. Clark also said he "would use AWS more if AWS could prove they have the level of security that we need."
Managing the source code is a big task. Clark said it's important to use tools that balance ease of use for the developers with the security and compliance needs of an enterprise. If a source code management system adds too much overhead for developers, they will rebel against it. Without the developers' buy-in, any source management scheme will fail. On the other hand, some easy-to-use systems trade security and control for usability; this is an unacceptable loss for most enterprise development efforts.
Clark still uses some tools on the lighter side of the spectrum. For example, his developers use Git for some quick and dirty projects that don't need robust source code management. On the heavyweight side, Clark said he was working on a project using IBM Rational ClearCase over a decade ago. He said "ClearCase was a huge, heavyweight system that could track hundreds of branches and weave them together in ways that -- now with componentization -- … are obsolete."
James Denman is the site editor for SearchSoftwareQuality and can be reached at firstname.lastname@example.org.
Follow us on Twitter @SoftwareTestTT.
Challenges and solutions for continuous integration testing
How to use test-driven development and continuous integration