End-to-end testing is a methodology that assesses the working order of a complex product in a start-to-finish process. End-to-end testing verifies that all components of a system are able to run and perform optimally under real-world scenarios. However, it's not accurate to claim that end-to-end testing is a form of testing a completed product. This is because there are other intricacies and forms of end-to-end tests.
Types of end-to-end testing
There are two types of end-to-end testing: vertical and horizontal. Horizontal end-to-end testing is the most utilized and well-known approach. Horizontal testing can build confidence in a system by assuming the perspective of a user. Horizontal testing confirms whether a user can navigate through a system, if it works as expected, and if there are any unexpected bugs or exceptions.
Vertical end-to-end testing refers to testing in layers, meaning that tests happen in a sequential, hierarchal order. Each component of a system or product is tested from start to finish ensuring quality. Vertical testing is frequently used to test critical components of a complex computing system which do not typically involve users or interfaces. A layer without a user interface (UI) would benefit from vertical end-to-end testing over horizontal.
How to perform end-to-end testing
End-to-end testing can be applied in various ways. For example, consider an e-commerce web application. Horizontal end-to-end testing may include a sign in to a profile, use of the search and browse functionality, add an item to a cart, save an item for later, buy an item, log shipping and credit card information, confirm the purchase, and then sign out.
By comparison, vertical end-to-end testing would be used in a system where there is no UI, or the UI has a high technical skill-cap. For example, it would work to test an application program interface or Structured Query Language interface.
Before testing, designers and developers construct a list of what UI or other functions they need to perform. Data flow should be tracked from system to system to find dependencies, flaws, or inaccuracies. Conditions should be prepared before testing, including factors that may change any interface options such as latency or traffic. End-to-end testing is typically conducted on finished products and systems, which makes each review a test of the completed system and how it communicates with its parts. If the system does not output what is expected, if problem or issue is found, then it is tested a second time. Data is recorded and analyzed to determine the issue's origin, and then it is fixed and re-tested.
Why use end-to-end testing?
End-to-end testing is often applied as a software testing methodology. Both forms of end-to-end testing simulate real-world scenarios like the interaction with hardware, networks, and databases. End-to-end testing assumes the advantages of white and black box testing. White box testing closely aligns with vertical end-to-end testing and examines the internal structures or inner workings of a program or system. Black box testing is comparable to horizontal end-to-end testing and focuses on functionality without examining the internal working structures of a program or system.
End-to-end testing can differ greatly from other software testing methods.For example, canary testing is a code testing methodology applied to a product already in production. New code, such as a patches or an update, is released to a small percentage of users who are unaware of this change. Canary testing enables a developer to continually test and make changes and improvements to a system while minimizing potential impacts of flaws or defects. If the new release works, the update can go out to a larger percentage of the user population.
Another form of software testing is A/B testing, also called split testing, where two versions of the same software are made available in parallel, allowing users to run either version in order to compare features, functionality, stability, performance, or other attributes of the software.
Other forms of testing include static testing and dynamic testing. Static testing is a method that examines a program's code and documentation without it being executed. Dynamic testing operates on the program itself while live to validate sections of the code.
Performing end-to-end testing vs. today's software testing
Software testing involves smaller tests performed at a faster pace – only relying on methodologies such as end-to-end testing for final integration and pre-release evaluation. For example, the Testing Pyramid illustrates when a test should and shouldn't be conducted.
At the bottom of the pyramid, the beginning of development for each system is unit testing. Unit testing addresses smaller pieces of the system or code and is ultimately faster and more efficient than end-to-end testing. If a bug is found in the overall system of an end-to-end test, it could be anywhere in the system. But if there is a bug found in a unit test, far less time is needed to find and correct the problem.
The middle section of the Testing Pyramid is integration, which tests how multiple units operate together with a proper flow of information. Integration testing is similar to vertical end-to-end testing in that it tests the underlying framework of the user interface.
The final and top portion of the Testing Pyramid involves a small amount of end-to-end testing (or UI testing) where the system is tested horizontally as a whole. If time is spent efficiently, most tests should be unit tests, followed by integration tests, then end-to-end testing.
End-to-end testing drawbacks
For all of the strengths that come with end-to-end testing, the practice is no longer widely used in software testing. Software test engineers can still choose to utilize end-to-end testing, but the practice has been supplanted by other testing methodologies.
Today’s software development cycles tend to employ some form of Agile development such as DevOps; a fast-paced and collaborative team effort approach to software development. End-to-end testing may be too slow for Agile methods of software development.