Getting a Handle on Software ROI, Part 1

A major auto manufacturer spends four years and more than US$200 million to install a Web-based enterprise resource planning system (ERP) from an industry leading developer, then pulls the plug and goes back to its legacy system.

An IT industry pioneer launches a leading vendor's customized order management system. Shortly thereafter, as many as 20 percent of the company's customer orders are stuck in limbo due to data transfer problems between a legacy order entry system and the new, back-end order management system.

The problems snowball as contingency plans to manage such unexpected events aren't sufficient to handle them. Result: Management, in a quarterly earnings conference call, tells analysts that actual revenue losses have amounted to $40 million -- more than the project's cost. In fact, including order backlogs, total losses are said to be well over $100 million.

Analysis Becoming Increasingly Difficult

Misadventures similar to these involving the installation and return on investment, or lack thereof, of major software projects are not uncommon.

IT and business managers put a lot of time, capital and effort into designing processes and methods aimed at giving them a firm, actionable understanding of what they can expect in the way of a return on their software investments.

Yet the increasing sophistication of enterprise software applications and database systems, together with the features these systems need to include and the tasks they must perform, make this job incredibly complex, especially in today's multi-networked, online business world. Never-ending mergers and corporate acquisitions that create ever larger organizations complicate the picture even further.

Some of the more recent software ROI concepts, such as Scrum, attempt to break a project down from a process management perspective. While more and more organizations and software vendors are committing to the Software as a Service business model, others, such as Forrester Research Principal Analyst Ray Wang and his team, are working with their clients to develop a broader-based extension of the Total Cost of Ownership (TCO) methodology for ERP.

TCO to TEI

The TCO approach is the best known and most widely used method of estimating software ROI. However, IT managers, software developers and process engineers have for years been experimenting with ways to broaden this methodology to better reflect all the likely costs, benefits and risks of choosing a particular software application over another.

Toward that end, Forrester late last month published a research paper that advocates adopting what Wang and his colleagues have termed the "Total Economic Impact" methodology of evaluating software ROI.

"Fundamentally, a pure cost-oriented approach like Total Cost of Ownership does not allow an organization to measure the full economic impact of the investment," Wang explained. "By measuring not just cost and financial benefits, but also risk or uncertainty, [and] future flexibility, enterprises can establish a more inclusive and accurate picture of the return on investment."

Using TCO is akin to showing only half the picture of a software package's ROI, Wang asserted. "You'd never know if increased cost investments yielded greater benefits."

The TEI approach is built around assessing four parameters -- cost, benefit, flexibility and risk -- and evaluating the relationships among them. Taken together, Wang said, the four parameters "provide an effective model for progressive IT business analysis, management and decision making when applied to a single project. They [also] provide the beginning of a potential portfolio analysis for IT expenditures as a whole."

"While cost calculations are easier to define," he added, "benefits, risks and flexibility can be simplified by applying rules of thumb and benchmark data to items such as implementation risk, error rates, training risks, etc."

An Elusive Goal

"There is an intensive and growing need to quantify the notion of Total Cost of Ownership, but as we all know, TCO requires an organization to look at all costs associated with a software project: licensing fees, implementation, maintenance and internal costs, etc.," explained Terry Blake, director of corporate communications at St. Paul, Minn.-based Lawson Software (Nasdaq: LWSN). "Obviously, some of this information is fairly easy to obtain, but much of the data -- particularly the intangible internal costs -- is extremely elusive for our customers."

Tangible costs such as software license fees can be more or less straightforward to predict, Blake noted, while intangible costs are hard to put a number on. "That's one thing people have liked about SaaS," he said, referring to its ability to cost-assess the intangibles.

Still, for software developers at the forefront of the SaaS trend and vendors using more traditional software service methods, delivering a project on-time and in-budget remains the keystone of evaluation success .

Read 'Getting a Handle on Software ROI, Part 2'