Stat Wars 2: A Tale of Two Surveys on Apache and IIS

Not long ago, TechNewsWorld published a story describing divergent conclusions reached by two recent studies of Web server market share.

One, the latest in an ongoing series conducted by UK-based Netcraft, cast its net wide and surveyed a sample of more than 45 million Web sites. It concluded that open-source Apache controls some 67 percent of that market, with Microsoft's (Nasdaq: MSFT) Internet Information Services (IIS) server a distant second at 20 percent.

In San Diego, California, Port80 Software begged to differ. According to Port80, "Netcraft's focus on sampling as many sites as possible introduces a systematic overrepresentation of servers that are used to host high numbers of sites, including parked domains (hostnames pointed to a physical server but not actively used as Web sites) and sites of extremely variable quality and traffic."

More relevant for business decisions, Port80 said, would be a more tightly focused survey that would exclude much of the statistical "chaff" inevitable in the Netcraft model. In fact, the company conducted two polls. One taken in January 2004 sampled sites belonging to the companies listed in the Fortune 1,000. It showed a far different picture than Netcraft's survey, with IIS garnering a hefty 53.4 percent market share, Apache 19.4 percent and Netscape Enterprise Server 17.1 percent.

This report generated considerable debate, and in order to, in its words, "add critical new insight to estimates of Web server market share," Port80 did another poll of what Nielsen//Netratings (Nasdaq: NTRT) considered to be the Web's 1,000 most heavily visited sites. This inquiry produced figures that, while more moderate, still contrasted starkly with Netcraft's findings. The Port80 survey showed IIS edging out Apache by a margin of 43.1 percent to 39.7 percent.

Asking the Right Questions

These very different results hint at the difficulty of getting a handle on anything as amorphous as the Internet -- and at the number of parameters that need to be clearly defined to create any meaning in a market survey. The first article in the Stat Wars series -- "Stat Wars: Measuring OS Market Share" -- alluded to several examples of the kinds of issues raised by these studies: "Are they tracking sales, or users? New orders or existing user base? Servers or desktops? Domestic or worldwide?"

A relevant subquestion in this case might be, "How old were the installed Apache and IIS versions?" It is a question that bears on upgrade behavior.

Reader letters in response to the TechNewsWorld story on IIS and Apache suggested that more indicative results could be obtained by using different sample groups -- "top government sites, top e-commerce sites, top Web hosting providers" and so forth. The reader letters also pointed out that results would change whether, for example, one considered Yahoo (Nasdaq: YHOO) -- which does not use IIS -- as one site or as many, such as tv.yahoo.com, groups.yahoo.com and mail.yahoo.com.

The letters also touched on the larger question, which has raised its head more than once recently, of analyst independence. Port80 maintains that Microsoft neither commissioned nor paid for the market share study, but the company is a Microsoft business partner and sells IIS-related products.

Some people remain unconvinced that objectivity is really possible under such circumstances, but at the end of the day, only an intimate knowledge of any analyst's internal company culture and business ties can provide a true picture of its level of editorial independence. That's information to which few of us have access.

Intertwined Trends

Joe Wilcox, a Microsoft specialist at Jupiter Research, said he sees two intertwined trends in the Apache-IIS debate. "So much is about marketing and positioning," he told TechNewsWorld. Wilcox estimates at least 35 to 40 percent of Windows Server installations are still NT 4. "Microsoft is trying to get those customers on older versions onto something newer, hopefully Windows Server 2003," he says.

At the same time, he said, Microsoft views Windows alternatives, especially Linux, as a threat. "Where the two things overlap," Wilcox noted, "is that many of those older systems are being used for single-function purposes, such as file and print, and also for Web serving. That dates from the days when Microsoft bundled IIS free with the Windows Server product. So those customers are also candidates to move to another operating system, like a Linux running Apache server.

"There's a lot of jockeying for position going on," he adds. "Microsoft wants to assure everyone that there's momentum behind Windows Server 2003, that customers are interested in the product, that they're going to move to it; that they're not going to hang around on NT 4. And, more importantly, that they're not going to go to something else like Linux."

And the Winner Is ... Marketing

But as with market-share surveys, there's more to cost than meets the eye. Wilcox notes that "your initial out-of-pocket expense may be much less with Linux and Apache Server because of how Microsoft does the client access licenses.... That might make Linux and Apache Server more attractive to a lot of smaller operations.

"Of course, there are many other things that go along with buying a product, in terms of service and support [and] how it works with other things you have in your organization," he added. "And those considerations can affect cost in other ways. So what looks the cheapest isn't always the cheapest. It depends what you want to do with it."

So who really is winning? In these conflicting reports, which survey methodology -- of big users or a cast of millions -- delivered the truest or most useful results?

Maybe it doesn't even matter. Joe Wilcox says that if he were an IT manager, he'd ignore most of the market share hoopla from both sides. "I wouldn't worry so much about what other people are doing," he says. "Just because 'the other guy' is doing one thing doesn't mean you should do it as well. You have to look at what's appropriate for your business."