In his new book, APM best practices: Realizing Application Performance Management, author Michael J. Sydor says the greatest potential for APM initiative failure is inadequate scope. In this first part of a three-part series of interviews, Sydor defines scope and gives advice to organizations on how they can make sure their APM initiatives are adequately scoped. The question of ROI with
SSQ: Let’s start by understanding the scope of the APM initiatives your book addresses. Are we talking about performance testing as well as post-production performance monitoring? Are these best practices for all size of business? All domain areas?
Michael Sydor: We are talking about the full application lifecycle: development, QA, user acceptance, production and triage. The visibility that APM technology and techniques supports has benefits for all these participants, which I simply call application stakeholders. They all contribute specific capabilities to bringing an application to operational readiness and keeping it there.
As for deployment scope, you will find practical advice for a single application all the way to an enterprise-wide deployment involving hundreds or thousands of applications. Smaller initiatives do not require the same planning and management as the larger efforts but every size of initiative will benefit from the testing and triage techniques.
The book is divided into Planning, Implementation and Practitioner’s Guide. This is to help accommodate initiatives that are new to APM as well as incremental efforts. I also make allowances for overall organizational maturity and define many of the techniques in terms of Good, Better and Best. This lets you decide the pace and scope of the best practices that suit the realities of your particular situation. There are many paths to start or continue an APM initiative.
SSQ: You say the greatest potential for APM initiative failure is inadequate scope. Can you give some advice on how organizations can ensure the scope of their initiative is adequately defined?
Sydor: It’s actually quite simple. Everybody plans for an APM deployment but very little consideration is given to how the technology will actually be used. The goal is often to "stand up" the technology and let folks figure out what to do with it on their own. This happens because APM "looks" very much like the system management technology that it augments. APM can generate alerts, just like other monitoring tools but the real capabilities diverge sharply from that point.
To both expose and explore the gaps an organization may have with an APM initiative, there is a self-assessment worksheet that you can use to understand the skills, processes and competencies that are the hallmark of successful APM initiatives. These are later developed into specific capabilities as part of a service catalog, which is the mechanism I use to adjust for the overall scope of an individual initiative.
SSQ: How are organizations able to determine ROI on APM initiatives?
Sydor: This is one of the more difficult topics that we face with APM. To show ROI you have to be able to quantify the costs of not using APM. It’s not as simple as a database upgrade or moving to a virtualization platform. To justify an initiative, I instead look at the potential reduction in severity incidents that APM visibility can address. Part of this is getting visibility where you had none prior but also a portion is because you can avoid deploy problems by addressing performance proactively. So if you know the full cost of a severe operational incident, then you can get a fair ROI. This also changes the initiative from monitoring and more towards software quality, which is a theme that most folks don’t expect.
However, what you miss in this analysis, which I provide a full example in the book, are the positive impacts on organizational efficiency that come from the collaboration that a successful APM initiative will foster. Improving cooperation and breaking down some of the organization silos that impede the flow of performance information -- these are difficult to quantify, even as they may represent major corporate goals. Software quality is a much broader initiative and includes these concerns but is often outside of the scope for the team focused on using APM for monitoring.
My typical approach is somewhat different because getting access to incidents and those costs can be difficult. I show you how to evolve an APM capability through short, low-cost iterations, with the simple goal of demonstrating a mutually agreed-upon value proposition. This takes advantage of the relative deployment ease of APM technology and moves the focus to demonstrating competencies -- showing how you can use the tools and processes successfully. Instead of trying to do ROI for a multi-year plan, you instead show what you can achieve in a 2-4 week iteration. It’s a welcome change to the conversation. The book provides a number of "cookbooks" to lead you through various competencies.
Michael Sydor is an engineering services architect for CA Technologies. With more than 20 years in the mastery of high performance computing technology, he has significant experience identifying and documenting technology best practices, as well as designing programs for building, mentoring and operating successful client performance management teams.
APM Best Practices: Realizing Application Performance Management is available at many leading book retailers, including Apress, Amazon.com, Barnes & Noble, Borders, Powell's, Safari, and Springer, among others.