Keyword-based software test automation can reduce the cost and time of test design, automation and execution. It allows members of a testing team to focus on what they do best, but also allows non-technical testers and business analysts to write automated tests. David W. Johnson explains how key-word based test automation works.
Keyword-based test design and test automation is founded on the premise that the discrete functional business events that make up any application can be described using a short text description (keyword) and associated parameter value pairs (arguments). For example, most applications require users to log in; the keyword for this business event could be "Logon User" and the parameters could be "User Id" and "Password". By designing keywords to describe discrete functional business events, testers begin to build up a common library of keywords that can be used to create keyword test cases. This is really a process of creating a language (keywords) to describe a sequence of events within the application (test case).
When properly implemented and maintained, keywords present a superior return on investment because each business event is designed, automated and maintained as a discrete entity. These keywords can then be used to design keyword test cases, but the design and automation overhead for the keyword has already been paid.
When a change occurs within any given keyword, the affected test cases can easily be found and updated appropriately. And once again, any design or automation updates to the keyword are performed only once. Compare this to the Record and Playback approach, which captures a particular business event or part of the business event each time a test case traverses it. (If there are 100 test cases that start with logging on, then this event will have automated 100 times and there will be 100 instances to maintain.)
Development of keywords should be approached in the same manner as any formal development effort. Keywords must be designed, coded, implemented and maintained.
The test designer is responsible for keyword design. At a minimum the design of a keyword should include the keyword name, keyword description and keyword parameters.
A standard keyword naming convention should be drafted and followed to allow designers to efficiently share keywords. The keyword name should begin with the action being performed
- Logon User -- <action> Logon <entity> User
- Enter Customer Name -- <action> Enter <entity> Customer <text> Name
- Enter Customer Address -- <action> Enter <entity> Customer <text> Address
- Validate Customer Name -- <action> Validate <entity> Customer <text> Name
- Select Customer Record -- <action> Select <entity> Customer <text> Record
The keyword name should be a shorthand description of what actions the keyword performs.
The keyword description should describe the behavior of the keyword and contain enough information for the test automation engineer to construct the keyword. For designers, the description is the keyword definition; for automation engineers, it's the functional specification. This should be a short but accurate description. Here is an example for the keyword "Logon User":
Logon User Description: On the Logon screen enter specified User ID and Password and then press the OK button.
The keyword parameters should capture all the business inputs that could impact the immediate business event being defined by the keyword. The simplest and most reliable method for getting the appropriate list of parameters is to take a "capture what is displayed" approach.
For the keyword "Logon User," the application displays three elements: "User ID", "Password" and OK button. The two parameters required to support this keyword are "User ID" and "Password." The OK button does not require a parameter because the keyword description states that the OK button will always be pressed. If there were multiple buttons, such as OK, CANCEL and EXIT, then a third parameter "Press Button" would be required and the keyword description would have to be modified.
The test automation engineer takes the keyword name, description, parameters, application under test, and keyword development standards and constructs the keyword. If there are any issues with the design aspects of the keyword, the automation engineer works with the test designer and the design is modified to clarify the intent of the keyword. If there are any automation/engineering challenges, then the automation engineer works with the development team and the tool vendor to find an appropriate automation solution that fits the automation framework.
Keyword implementation follows the same path of any shareable project resource. At a minimum, the completed keyword should be reviewed by the test designer, unit tested by the automation engineer, function tested, and migrated into the project "testware." This does not need to be a complex or extensive process, but it should ensure that any implemented keyword is published to the testing team and functions as expected.
Keyword maintenance occurs when a keyword defect is detected, when a business event changes or a when keyword standards are modified. Keyword maintenance follows the same deployment path as keyword development: design, code and implement.
Keyword test case
Keyword test cases are a sequence of keywords designed to test or exercise one or more aspects of the application or applications being tested. Keyword test cases must be designed, executed and maintained. Keyword test cases are the responsibility of the test designer/tester. The automation engineer becomes involved only if a defect occurs during keyword test case execution. It should be noted that the keyword design paradigm is often used in the absence of keyword automation. It is an effective standalone test design paradigm.
Keyword test case design involves planning the intent of the test case, building the test case using keywords and testing the design against the application or applications being tested. At first glance this does not appear to be any different from any other method for test case design, but there are significant differences between keyword test case design and any freehand/textual approach to test case design.
Keyword test case designs are as follows:
- Consistent -- The same keyword is used to describe the business event every time
- Data driven -– The keyword contains the data required to perform the test step,
- Self documenting -- The keyword description contains the details of the designers intent
- Maintainable -- With consistency comes maintainability, and finally the ability to support automation with no transformation from test design to automated script.
Test designers gain the power of test automation without having to become test automation engineers.
The tester can perform keyword test case execution manually by performing the keyword steps in sequence. This should be performed as part of the keyword verification process. Test cases, which are constructed using automated keywords, can be executed using the test automation tool or by an integrated test management tool. Test case execution should always be a mechanical exercise whether automation is in use or not. The test case should contain all the information necessary to execute the test case and determine its success or failure.
Test case maintenance must occur when changes occur in the application behavior or in the design of a keyword that is being used in one or more test cases. A properly implemented keyword framework will allow the tester to find all instances of a keyword via some query mechanism, reducing the often-painful process of finding the impacted test cases to one simple step. Furthermore, a well-implemented keyword framework should support global changes to keyword instances.
GUI (graphical user interface)
Keyword solutions for GUI-based applications are the easiest to understand and implement. Most shareware, freeware and commercial applications of keyword testing deal with this space.
API (application programming interface)
Keyword solutions for API-based applications appear more complex on the surface, but once these applications are broken down into their discrete functional business events, their behavior is much the same as an equivalent GUI application.
If the business event were "Logon User," it doesn't really matter what application mechanism is used to implement the event. The keyword would look and behave the same if the business drivers were the same. There are several keyword solution vendors that deal with the API space, and the same vendor often has a solution for GUI applications.
Telecom (Telecommunication Protocols)
Keyword solutions for the telecom space (example SS7) require an intimate understanding of telecommunication protocols. There are vendors that offer keyword solutions in this space.
Keywords and test phases
Keywords can be applied to unit tests, but it is not recommended. The development group, using the tools and techniques available in the development suite, should do the unit tests.
Function (integration test)
Keyword test solutions focused on designing and implementing keywords as discrete functional business events offer one of the most cost-effective and maintainable test frameworks for function tests. In fact, if test automation of a GUI- or API-based application is required or desired, there are few frameworks that can match its short-term or long-term ROI.
A keyword-based testing solution that leverages the keywords from function test to the system test phase will help expedite the testing process. An effective keyword framework will allow the test designer to combine function-level keywords into system-level keywords.
System-level keywords deal with complete business events rather than the discrete functional business events that make up a business thread. For example, a system-level keyword could be "Complete Customer Application." And that could be made up of this chain of function-level keywords: "Enter Customer Name," "Enter Customer Contact Information," "Enter Customer Personal Information" and "Save Customer Record."
User acceptance tests
Keywords can be applied to user acceptance tests, but it is not recommended unless this is an extensive phase of testing. The end user community using the tools, techniques and processes available in production best perform user acceptance tests.
About the author: David W. Johnson is a senior computer systems analyst with over 20 years of experience in IT across several industries. He has played key roles in business needs analysis, software design, software development, testing, training, implementation, organizational assessments and support of business solutions. David has developed specific expertise over the past 10 years on implementing "Testware," including test strategies, test planning, test automation and test management solutions. You may contact David at DavidWJohnson@Eastlink.ca.