In my previous article I reviewed the various roles within a Test Center of Excellence. Let's take a closer look now at what a test automation engineer or test engineer does and how he works with other people on the team.
A role-based approach to testing enables your testing organization to have clearly delimited skill sets, responsibilities and career paths. And it allows you to implement appropriate reimbursement. There is an essential core set of roles that are required to effectively support any testing organization:
- Test manager (or test lead)
- Test designer (or system matter expert)
- Test automation engineer (or automation engineer)
Larger testing organizations will require additional roles to effectively support the organization:
- Environment support (lab "rat")
- Test architect
- Test methodologist
The role of the test automation engineer (test engineer) is to design, build, test and deploy effective test automation solutions. To fulfill this role, the automation engineer applies appropriate automation technologies to meet the short- and long-term goals of the testing organization. The objective is to automate as much of the testing effort as possible with a minimum set of code/scripts. The focus should be on the test effort not testing coverage. For example, if one manual test case or manual test preparation process consumes a large percentage of test resources, then this manual process should be the first to be automated.
Responsibilities and deliverables
The responsibilities and deliverables of a test automation engineer are dependent upon the test phase and test automation framework.
Test phase
Unit test
The test automation engineer, in this case the application developer, instruments the application code in order to enable effective and repetitive unit testing of the code before it is incorporated into the current build. There are several Agile development paradigms that incorporate this concept/process into their standard development cycle.
Function and system test
The test automation engineer develops automated solutions to expedite testing. This can take the form of tooling to increase the efficiency of test preparation and/or the creation of automated test cases.
Acceptance test
The test automation engineer develops and deploys automated solutions to expedite acceptance testing. In function and system test, the test engineer fulfils the same role, but it is in the context of the test organization. If automation tooling is deployed as part of acceptance testing, then the automation solution should be treated and tested as part of the system being deployed.
For any given test phase, the objective of the test automation engineer is to put the power of automation into the hands of test designers/testers. The test engineer should deploy the simplest solution to meet the defined need. The objective is not to build the "best automation solution ever" but to effectively automate the testing effort.
Test automation frameworks
A test automation framework is the method or process being used to implement automation. Several frameworks have been implemented over the years by commercial vendors and testing organizations. The skills required to develop within any given framework define the skills required by the test automation engineer.
Record and playback
Record and playback frameworks were the first commercially successful testing solutions. The automation engineer simply records a series of steps or actions against the application using the appropriate record and playback tool.
Extended record and playback
It quickly became apparent that a simple record and playback paradigm was not effective and did not make test automation available to non-technical users. Several commercial vendors, test organizations and automation engineers began extending the record and playback framework to make the solution more robust and transparent. These extensions include data-driven, keyword-driven, and component-based solutions. As the record and playback framework was extended, the automation engineer had to develop and deploy solutions (code) within the context of the extended framework.
Keyword-driven
Recent innovations have lead to commercial keyword-driven frameworks -- as opposed to extensions of the record and playback paradigm. This gives non-technical users access to the power of automation without having to become a developer, and it allows the automation engineer to focus on automation as opposed to construction of automation tools.
Load/performance
Load/performance test frameworks provide a mechanism to simulate transactions against the application being tested and to measure the behavior of the application while it is under this simulated load. The automation engineer determines how to load, measure and control the application. Using the load-testing tool the engineer implements the automation solution to accomplish this task; the Automation Engineer will often need to extend the load/performance framework.
This is not a complete list of the test frameworks available, but it illustrates how the skills and abilities required to fulfill the role of test automation engineer depend on the framework.
Testing mandate and scope
The test automation engineer must have a clear understanding of the testing mandate and how automation can be applied to help meet this mandate. The temptation of any automation engineer is to automate everything; the challenge is to determine what should be automated and in what sequence to get the maximum return on the automation investment. Other roles in the testing organization focus on testing the application. The automation engineer attempts to expedite the testing process by supplying and maintaining an appropriate automation solution.
Relationships with other team roles
Test lead/manager
The test engineer must have a good working relationship with the test lead, but more important than that, the test engineer must keep the test lead aware of any challenges or costs that could impact the delivery of test automation. The relationship between the test lead and automation engineer is similar to the relationship between a development lead and developer. The automation engineer must size, implement, and test a solution while the lead ensures the goals of automation are met.
The one significant difference between traditional development efforts and test automation is that there is always an option not to automate. In fact, if automation cannot show an immediate return of 4 to 1 in terms of effort/hours, then it probably does not make sense to proceed with the automation effort.
Test designer/tester
The test automation engineer must understand precisely what the test designer wants any given test case or group of test cases to accomplish. If you think of test cases as requirements, then the test designer defines the requirements for the automation effort, and the test automation engineer implements those requirements. In many ways, the test automation engineer works under the direction of the test designer in terms of what automation needs to be built, but it is the role of the engineer to determine how it should be built.
Developer/product support
The test automation engineer will often encounter automation challenges that require an intimate knowledge of the application being automated. In order to have any chance of building a sustainable solution, the test automation engineer must have a close working relationship with development or with product support if the application has been purchased by a third-party vendor.
-----------------------------------------
About the author: David W. Johnson is a senior computer systems analyst with over 20 years of experience in IT across several industries. He has played key roles in business needs analysis, software design, software development, testing, training, implementation, organizational assessments and support of business solutions. David has developed specific expertise over the past 10 years on implementing "Testware," including test strategies, test planning, test automation and test management solutions. You may contact David at DavidWJohnson@Eastlink.ca.
