E-BUSINESS SPECIAL REPORT
Web Performance Metrics That Matter

As recently as the late 1990s, a simple counter provided the only metric most Web site operators were interested in -- how many people had visited their sites.

However, as e-commerce has matured and evolved, so has Web performance measurement. Site metrics are now a complex -- although essential -- proposition, telling a company not only how often users cannot reach its site fast enough to make purchases, but also how well it is performing compared with its competitors.

"With more customers moving to broadband, companies need to know their sites are available and responding the way they're supposed to," John Klinke, senior product manager at Web performance measurement firm Keynote Systems, told the E-Commerce Times. "The user's experience is going to determine if they're going to continue to use the site as a channel to interact with that company or not."

In an era when customers increasingly are judging offline companies on the baiss of their online offerings, the stakes are incredibly high. More and more companies are tracking Web site metrics -- and striving to boost performance on the basis of what they learn. What metrics should companies be measuring -- and how can they take action on their findings to boost the bottom line?

Vast Differences

Although advances in technology and plummeting prices have put high-performance Web sites within the reach of most big-name companies, there are still substantial differences in how various sites respond to customer demands. Some sites, which run scores of different applications intended to work together seamlessly, can experience slowdowns even if they perform rigorous testing.

Even among top sites, performance differentials can be significant. For example, in Keynote's August 18th e-commerce rankings, the site with the fastest aggregate response time -- EddieBauer.com -- responded in an average of just over 6 seconds, while number-two site Amazon.com took about 11 seconds. Office Max occupied tenth place in the rankings, with an average response time of more than 24 seconds -- nearly four times that of the best performer.

While such rankings are helpful, most companies want to know mainly how they stack up against their direct competitors. In other words, Amazon may be more likely to compare its performance with Wal-Mart's than with Costco's.

"To really understand how your site is doing, you have to understand how your competitor is doing," Klinke said. "Even if your customer doesn't go over there halfway through a transaction, he might remember next time how long it took on your site."

How Much Is Enough?

Frequency of measurement is another consideration for companies weighing the costs and benefits of Web site performance metrics. Once per hour is a fairly standard approach, although Keynote said many of its customers have their site performance checked every five minutes or even more often.

Once a company decides to measure its Web site performance, it also must determine how to use that information to improve its business bottom line. In each company, more than one department might be able to turn basic metrics into valuable information. For example, the head of marketing can tell whether a sales promotion is working, while the IT department can pinpoint problem areas.

The popularity of Web performance metrics seems to be on the rise. In addition to the financial services and retail firms that traditionally have made up the bulk of its business, Keynote said it is gaining more public-sector accounts as state, federal and local governments seek to boost online use. "They deal with a lot of the same issues -- they want people to come back to that online channel in the future," Klinke said.

Different Strokes

Within any organization that relies on the Web, different groups will want to measure different metrics, said Carol Carpenter, director of product marketing at Keynote.

She explained that the goal of Keynote's suite of products is to give IT departments key information to diagnose slowdowns or other problems and quickly deploy solutions. The company's diagnostic suite measures the speed and success of transactions and enables companies to perform on-demand load testing and other diagnostics.

The company also offers tools geared toward marketing departments that measure Web site performance compared with competitors and help quantify the impact of online performance on revenue. Yet another offering is aimed at Web development departments and includes tools for live-testing sites before they are made available to the public.

The Whole Story?

However, while metrics are essential -- and likely to become more complex, now that establishing better links between online and offline commerce is an important goal of most multichannel retailers -- even the most detailed statistics may not tell the whole story.

A report commissioned by TeaLeaf Technology, which sells Web application management and measurement services, found that even sites listed as top performers on indexes like those from Keynote and Gomez often have hidden weaknesses. The report, written by analysts at the Business Internet Group and unveiled by TeaLeaf at the eTail 2003 conference last month in Boston, found that these undetected problems can cost businesses hundreds of dollars per shopper.

Sales-sapping glitches are often hidden deep in applications, TeaLeaf director of marketing Tim Smith told the E-Commerce Times. "Shoppers might break off a purchase and give no indication of why," he said. "Every Web site is actually a composite of dozens of applications, so tracking what's really going on with all of them is important.... Metrics are important, but the human element will always have a place as well."

In fact, most companies find success by combining a variety of approaches, including third-party monitoring and regular focus groups for usability testing.

"The problems might be in some place you never thought to look," said John Nordmark, CEO of eBags.com.

Go on to Part 2.