cutimage - Fotolia
Functional testing is the largest piece of the software testing puzzle. The QA process is inherently necessary and should result in a higher quality product that customers feel positive about.
Functional testing's goal is to confirm that software performs as described in requirements or acceptance criteria. This type of testing also validates that the software meets customer expectations.
While functional testing is the most executed type of software quality assurance, it actually comprises several different parts, each with a distinct focal point. Each functional testing type aims to determine whether an application meets needs -- on the customer front end and on back-end processing engines.
Let's discuss functional testing types, their objectives and order of occurrence. We'll also use functional testing examples to guide you through the process, all through the lens of a sample application. This article covers, and includes tests cases for, the following types of functional tests:
- Feature tests
- Unit tests
- User story tests
- Integration tests
- Interface tests
- Smoke tests
- Regression tests
- User acceptance tests
The fictional example application is called Return Me to Work, and it includes a web portal and a mobile app. Human resources personnel use this web portal application to manage employees who are returning to work after an extended leave due to a contagious illness, and track their exposure to other employees. HR can use the web portal to view each employee's daily updates and monitor any issues, as well as assist employees with medical, financial and other care needs. Employees use the mobile app version, through which they can fill out a daily survey form to check in, and upload any necessary medical documents. The survey responses automatically upload to the web portal for HR use. The employees keep their address and contact information up to date.
Feature, unit and user story tests
The majority of functional testing occurs when the QA tester evaluates a new application feature. There are three primary types of functional tests that evaluate application functionality in this manner:
- Unit testing. With this type of test, QA engineers evaluate the smallest possible segments of code. QA engineers can write manual unit tests or use test automation to execute them when developers check in the code or compile it for a test build.
- Feature testing. These tests assess the code update or new feature from the user's perspective. Testers interact with the software to see how well the code works and where to improve upon the version.
- User story testing. These business-oriented tests help organizations assess whether the feature was delivered as expected within the context of what the user expects.
These three types of functional tests all verify the app's functional adherence to requirements or acceptance criteria. QA engineers check each individual piece of a system under test as a single story or a single feature; they don't worry about how one component interacts with others or the whole.
Here's a functional test example for the Return Me to Work app. The development process might cover uploading exam or lab results when submitting the daily survey via the mobile app.
Unit, feature and user story test case
Test scenario: Send survey with uploaded lab results.
- Log into the mobile app with valid employee credentials.
- Select each answer from the drop-down list for:
- Overall health
- Select the Upload button.
- In the Upload window, select a file to upload in PDF format.
- Click the Submit button.
- The Review Submission window opens.
- Click OK to upload results.
- Make sure the employee receives a confirmation email that the lab result has been successfully received by the HR system.
- Select the Complete Survey button to submit all survey answers.
- The Review Submission window opens.
- Click OK to submit daily survey data.
- The Review Submission window opens.
- The employee returns to the landing page of the daily survey.
- Daily Status displays as Completed.
- Log out of the mobile app.
This functional testing example goes through the process of uploading a PDF document when filling out the daily health survey. When both actions are successful and no errors pop up, then the test passes.
This testing technique is a way to assess how one feature interacts with others within the application workflow. With integration testing, the QA engineer checks that the feature affects other components as expected.
Integration points within an application vary, and they can be complex. It's not good enough for app functionality to work in isolation. Applications with complex functionality or test scenarios for integration can make these tests difficult to execute. For example, many healthcare-related applications have both healthcare and financial functionalities. An application might send a prescription to a patient and generate a billing charge by connecting a diagnosis with a financial application code. All of these individual components must synchronize and pass data between each other for the application to work.
For Return Me to Work, an integration test case might include scheduling an appointment, following up afterward and a medical bill sent to a patient's insurer. In this test scenario, an employee has tested positive for a virus and has seen a doctor for a follow-up consultation or exam. Now the employee needs medication, and for the app to bill the employee's health insurance for both the exam and prescriptions.
Integration test case
Test scenario: Diagnosis and follow-up medical exam and billing to patient's health insurer.
- Log into the mobile app with valid employee credentials.
- Fill out the questionnaire indicating symptoms.
- The app schedules an appointment with the employee's medical provider.
- The patient attends the appointment and uses the app to indicate diagnosis by uploading the viral test document.
- The employee uploads a note or report from the provider.
- Once HR receives the data, the employee is placed on a work hold and the following occurs:
- A bill is sent to the employee's health insurer for the medical appointment and test.
- The application lists the employee on a work hold for seven days and then resends the daily health questionnaire.
Integration testing is complex. No one step in this test case means the test passes. Instead, this testing technique ensures that each action, such as the employee uploading the medical update, triggers the application to do the appropriate follow-on steps. As a tester, I not only check this workflow, but also functional aspects of the interaction. For example:
- Did the software system generate an accurate billing statement for the prescription, test and medical appointment charges?
- Did the system generate billing invoices and send them outbound to the health insurer?
- Was the prescription order correct?
- Did the system record the employee's medical test results with date and time, then set the patient to a hold status for seven days?
- After seven days, did the employee receive the daily health questionnaire?
This scenario includes testing connected APIs that send data outbound, as well as financial statements and calculations. This testing effort is integral to delivering a quality, fully functional software system.
With interface testing, QA engineers evaluate APIs or other back-end data exchange connections. Some experts equate interface testing with integration testing, but it is distinct because of its complex nature. Modern web and mobile applications use APIs to exchange inbound and outbound data. Numerous open source and proprietary tools test REST and SOAP API functions.
For interface testing, don't simply verify that API endpoints are functional; also test whether they can receive and send data securely. Test that API security is set up to only allow and connect to authorized vendors or partners.
Interface testing should also examine the data that flows through the connection. Confirm that the correct data structure is received, and that it is valid. Loop in the development and security team to help set up valid interface tests.
As an interface testing example for Return Me to Work, look at API endpoint security in the feature that sends prescription orders to the pharmacy.
Interface test case
Test scenario: Send prescription order to pharmacy for specific patient.
- HR personnel logs into the web application and confirms the employee submitted their medical appointment forms, including a prescription order.
- HR receives a confirmation response that the pharmacy received the prescription for the specific employee.
The tester should view the API requests and responses to verify the security token was passed when the app connected to the pharmacy. Confirm that the prescription information -- including patient name, address and medical provider -- is accurate throughout the exchange.
To create a good negative test scenario, introduce a fake pharmacy that doesn't have secure credentials. Let that pharmacy try to connect to the API to receive prescription data. The application should not send information. See what happens. Does the system send an alert to the IT group? Does it lock down the entire application? Or, worse, does it allow the connection?
Smoke testing, also called sanity testing, aims to ensure the application is in a proper state before more formal testing begins. Through smoke tests, the QA engineer checks whether the application functions as expected. Smoke tests are useful when a small or emergency code release occurs. For example, a customer reported a critical error that developers fixed as quickly as possible. QA must confirm the fix did not break any other existing functionality -- in a short time frame. When critical defects lead to an unplanned development project, smoke or sanity testing makes sure the application functionality is not adversely affected.
QA engineers often use automated smoke test scripts, but they can also be short, manual test suites. Most smoke test suites have an execution time between one and two hours.
Let's include a smoke testing example for Return Me to Work.
Smoke test case
Test scenario: Start on the login page of the HR web application, and then process employee health tests and appointments.
- Log into the web application as a valid HR user.
- Select the record of an employee who has reported feeling ill in their daily survey.
- Generate an illness test and set up a medical appointment with the patient's provider.
- Generate a call reminder to check in on the employee's condition.
- Review medical documents, and then mark them as Reviewed and Approved.
- Select to send out billing invoices to the health insurer for qualified medical charges or appointments.
- Change the patient work status to On Hold.
- Update the patient work status to Active, after all medical requirements are met.
The tester must determine which functions are critical to the customer when devising a smoke test. In our application, the HR personnel must monitor employees' medical evaluations, schedule appointments and ensure billing statements or pharmacy orders get processed in a timely manner. The HR role also ensures the employee can return to an active work status.
Regression testing occurs just prior to a planned release. QA engineers rerun tests to make sure the changes haven't adversely affected the application.
Regression testing varies in execution time depending on the development methodology; it can be as short as one day or as long as two weeks. The regression test suite includes all of the functional test scenarios, including integration, interface and smoke tests.
The time allotted for execution and test suite size determine how much regression testing a team can accomplish. As the application ages, more and more functionality tests accumulate. There comes a point where the test execution is too much for the QA resources and time allowed. When that happens, it's critical to plan your regression tests. Base a regression test plan on which functional areas experienced changes within the release, or which are the most critical to the customer base. One method is to create multiple suites of regression tests that cover all the functionality, and rotate their execution. For example, if you split the regression test suites into a few sets, execute one set per release.
In theory, you want to test everything, all the time, but that's not realistic for many application development teams. If you can't execute all regression tests at once, then plan to continuously execute them, spreading out the task to ensure coverage.
Regression test case
Test scenario: For Return Me to Work, a regression testing suite includes all the functional testing examples executed during feature, integration, interface and smoke testing. In this example test suite, follow the same test steps to reevaluate those above scenarios:
- Send survey with uploaded lab results.
- Confirm viral test and follow-up medical exam, prescription and billing to patient's health insurer.
- Send prescription order to pharmacy for specific patient.
User acceptance tests
User acceptance testing (UAT) is less about finding defects or failed requirements, and more about meeting customer expectations and improving user experience. For that reason, with UAT, the testers aren't QA professionals. Rather, this testing approach typically relies on the product team or representatives from the user base who validate that the software performs as they expected.
UAT frequently uncovers missing functional requirements or pieces of a full workflow. UAT reveals when requirements or acceptance criteria are poorly understood or communicated between the customer, the product team, development and QA. If the functional requirements fail to fully account for customer expectations, it's likely the final product will fall short for the user. For this reason, the customer and product team must fully understand the business workflows and objectives involved with the software project and ensure that the requirement specification, acceptance criteria and development stories are created accurately.
UAT often occurs at the end of a sprint, in the form of a demo. Sprint demos to customers and product teams help ensure no requirements were changed or missed during the software development lifecycle.
If possible, perform UAT either before or after regression testing. UAT can also occur during regression testing to save time, but that approach can mean defects get reported multiple times. If you execute regression and user acceptance tests at the same time, make sure QA and UAT testers collaborate to avoid creating duplicative work.
UAT suites typically assess real user scenarios. Here's a UAT example for the Return Me to Work app.
UAT test case
UAT suite scenario: Complete employee forms and verify they're received.
- Log in as an employee to the mobile app.
- Fill out the daily health questionnaire with valid data.
- Upload daily morning temperature reading.
- Sign certification that you feel healthy and able to work.
- Submit questionnaire and certification.
- Confirm receipt of the questionnaire and certification by logging into the HR web portal application and viewing the patient record.
Functional testing covers a lot of ground, with each type of functional testing designed to confirm an application performs as expected once it reaches the customer. There are even subsets of the functional testing types described above. For example, boundary value testing can fit into feature testing and integration testing.
The beauty of functional testing lies in flexible test coverage. You can test endlessly or check something quickly. Both approaches can provide quality test coverage depending on the needs of the application and its customers. However you approach functional testing, make sure to assess the critical functions that keep the application secure and performing as expected, release after release.
More than functional tests
Non-functional testing is also a major component of software quality success. QA engineers perform non-functional testing via load, stress and other forms of performance testing. Non-functional tests also include compliance testing, security testing and, in some cases, accessibility testing.