Which APM tool is right for your organization?
A collection of articles that takes you from defining technology needs to purchasing options
Application performance management (APM) is a discipline that includes all the tools and activities involved in observing how software and hardware are performing. These tools present that performance information in a form managers and developers can use to make decisions. This article covers some of the most important aspects of APM, and how APM software can help enterprises get the latest information on how their products are performing.
Dimensions of APM
Here are a few important aspects of application performance management to keep in mind. Each of these categories is a potential source of information about user experience, software performance and where to look when something goes wrong.
Users and transactions:
The best way to learn about how someone wants to use your software is to spend time with them and observe how they work. With software running in the cloud, that isn't always possible. Users can be located all over the globe, want to use your product at different times of the day and value different things. Collecting information about your users can expose where your customers are located, which pages they hit most often and in what order, and how long it takes them to send and receive data.
Breaking software into microservices, or an API, is probably one of the most popular modern software architectural choices. One benefit of this architectural style that software developers and ops people are discovering, along with the development of APM, is that microservices let you monitor specific pieces. Individual aspects of a product, like the user authentication service, can be monitored instead of the software as a whole.
The most precisely optimized software in the world won't perform well on an under-powered machine. I was working on a tool for sales people to create deals and the test environment was constantly lagging. We would start the morning with a fresh environment and as the day went on and the environment got busier, things slowed down fast. Some investigation showed that we didn't give enough memory to the Java virtual machine. Hardware monitoring will give you a stream of data on server resources, such as available RAM, CPU usage and disk space usage, so you can tweak the environment before things get too slow.
Reporting is probably the most important aspect of APM. Data that you don't have access to, and that you can't shape into different formats and views, isn't that useful. There are many different ways to deliver information about application performance. The simplest is a stream of real-time data displaying information about your servers and virtual machines, as well as application data. I like to have this running on an extra monitor to scan when I feel the need. Another popular reporting style is based on triggers, specific words or events, which result in emails and text messages being sent to operations and developers (and sometimes managers) to let them know something bad might be happening.
Where is the value in APM software?
Part of the value of application performance management tools is that you can get the functionality of three or four different tools built into one product.
The old way of getting performance data was a several-day event. First the performance team and a few developers would spend time figuring which specific scenarios they cared about, then they'd record those scenarios using a tool like JMeter, then get some custom scripting to handle the complex bits, and then finally run the script to see what happens. All of that to collect one set of data for one scenario. Most people will want to see more -- number of users, data size and server resources. APM software lets you get a constant flow of real-time performance data. Instead of getting data once in a while when the scenarios are run, we can almost immediately see the performance implications of a change.
Here are some performance metrics that testers collect with APM tools:
- Number of processes
- Number of simultaneous queries
- Number of simultaneous users
- Number of pages served per second
Using metrics can be tough when you have only one set of data. A set of data may reveal that there's a six-second page load time, and that might seem bad, but how would someone know unless they had another set of data to compare it to? Seeing all of your data plotted in a graph over a few days, weeks or releases adds a much needed perspective. APM tools let software practitioners lay different sets of data next to each other to help identify trends that businesses can act on.
Monitoring production environments is probably the most common way to use APM tools. The insight into what your software performance looks like at all times is valuable. When you start seeing exceptions and errors in the application logs, it's too late. Your customer is experiencing problems. One way to scoot this discovery back is to use APM on development and test environments in addition to production. Instead of searching through log files trying to find the exact moment an error was thrown, you can watch it on a second monitor and see the error, or resource spike, or load time lag, in real time.
Pulling people together
There are a lot of technical underpinnings for APM, but ultimately, like requirements, stories and test charters, APM software is focused on improving communication. APM products help developers and personnel peek into what the customer is experiencing, help testers show developers what is happening at a technical level in the software when they perform certain actions and help developers show the importance of a needed change to their managers.
Here are the top three objectives to keep in mind when testing application performance.
Your APM software is only as good as the metrics you choose.
Running on the cloud? There are APM tools specifically designed for cloud apps.