E-tailers getting their sites ready for Cyber Monday, the so-called unofficial start of the holiday buying season for online shoppers, know there is a lot at stake. They don't want poor application performance to be the cause of lost opportunity, which is why online stores like Zappos and Backcountry are utilizing application performance management/monitoring (APM) solutions to root out issues in both pre-production and production...
That opportunity, even during last year's recession economy, is potentially lucrative. According to Forrester Research, nearly three-quarters of online holiday buyers maintained or increased spending in the online channel in 2009 compared with 2008. Overall, Forrester reports that U.S. online retail (excluding auto, travel, and prescription drugs) grew 11% in 2009 to $155.2 billion, and expects that total to reach $248.7 billion by 2014.
So e-tailers can't afford sluggish performance. "Customer conversion rate is a make-or-break situation," said Ryan Quinn, senior engineer at Zappos.com, a shoes and apparel e-tailer. "Every half-second your customer is waiting for a screen to load, the conversion rate is plummeting, especially when you're looking at five times traffic [during the holidays]."
"In e-commerce, things are immediate," said Kelly Phillipps, CTO of Backcountry.com, an online retailer of outdoor and backcountry gear. "When money stops [coming in from customers] for two minutes, we know something is wrong."
Traditional monitoring solutions that focus on network uptime, CPU usage, etc., don't provide insight for what's happening at the application level, Phillipps said. "I've found it's best to look at what customers are experiencing—how long it takes to hit a 'submit order' button and get a thank-you back tells me more about my system than a thousand monitor points. This is not about trying to get a customer experience, but about how I'm meeting service levels to my customers."
Both Backcountry and Zappos use APM solutions to detect and troubleshoot application performance. Backcountry has implemented Foglight from Quest Software Inc., Aliso Viejo, Calif., which the e-tailer uses in both QA and "on the wire" in the production environment. Zappos is using dynaTrace from dynaTrace Software Inc., Waltham, Mass., in its pre-production lab.
Zappos brought in dynaTrace last year, when it was building the lab. "We'd go down during the holiday season; the amount of shopping would bring us to our knees," Quinn said. "Our goal was to build the lab well before the holiday hit," which was in the September-October time frame of 2009.
Quinn said Zappos uses a lot of open source, and during the proof of concept phase with dynaTrace "we found right off the bat that third-party tools were throwing an enormous amount of exceptions on non-exception events."
Once the lab was built, dynaTrace also helped Zappos identify that it needed to rethink how caching was done on the page. "Our current method was poor—a lot of garbage collection during runtime, which is no good, causing latency." After rebuilding all the caching, Quinn said they saw the effectiveness "immediately."
According to Carl Eberling, vice president and general manager of virtualization and monitoring at Quest, a common problem organizations often find immediately with an APM solution is "the amount of hops a transaction will take. When you start looking at the transaction and understand the amount of roundtrips a single transaction can strike off on, it's probably the biggest eye-opener."
Toffer Winslow, CMO of dynaTrace, said other top problems with application performance are "chatty remoting services; different tiers of the application talking to each other in inefficient ways." Also, he said, the use of Ajax and frameworks like JQuery can "create blind spots. You have no visibility in the last mile, the end user's browser." He said the Ajax edition of dynaTrace can provide a deep view inside the browser.
Blind spots and sluggishiness for e-commerce companies "is death," Winslow added.
So for those e-tailers tuning up for Black Monday, here is some advice from the trenches:
Batten down the hatches: "It's time to set aside innovation for a few weeks and batten down the hatches," Phillipps said. "Find all those slow and messy things and take care of the little things, so that when big things come it will take care of it itself."
Identify high-volume transactions and prioritize: After inventorying, revisit SLA targets, advised Eberling. "If your target for a checkout procedure is a minute, how many transactions are going over a minute?"
Analyze end-to-end: Once you identify a priority transaction that violates the SLA, "step through it procedurally, until you get through the application in its entirety. The challenging part is being disciplined about it," Eberling said.
Do realistic load testing: "Leading up to the holiday shopping season, get a good baseline of the traffic they should expect based on history," Winslow said.
Get insight: "You need real insight into the issues the load testing products uncover," Winslow said. "It's not enough to say the application fell over; you need to know why. You need solutions that give deep visibility into the issues the load testing uncovers."
Rethink SLAs: "Just looking at 99.99% availability is not the best bellwether," Eberling said. Rather, "how many new customer registrations do I want to do in 15 minutes?"
Listen for the silence: "Most of these [traditional monitoring] systems listen for noise, for screeching tires and crunching metal," said Phillipps. "We listen for silence; if we stop hearing what we expect to hear, we react to that."
If you're shopping for an APM tool, demand proof: "You don't have to commit to anything to do a proof of concept," Quinn said. "We found benefits immediately [with dynaTrace]. My advice is: don't make a decision on a tool unless they can demonstrate it will work."
Your goal is to be bored: Do the application performance tuning ahead of time, Phillipps said, because "you need it to be quiet so you can be responsive. You don't want to have incidents, but you probably will and you will wish you were fresh."