The fact is, nobody believes in ROI. Its the most manipulated figure on earth, argues Keith Ellis, principal at Digital Mosaic, a business consulting firm. In the companys recent white paper The ROI of ROI, he cites research showing when companies put together business cases, very few of these casesas few as 5 percentactually project a negative ROI. This doesnt surprise Ellis: If you go to a CFO and say the ROI of a project is 60 percent, would you get funding? Nearly every investment has a hugely positive ROI, but how many have paid off?

Most insurers that claim to take an ROI-focused approach to technology investment arent telling the truth, Jakubic adds. Ive seen many surveys where people are asked about how many decisions are subject to ROI, how often they [calculate ROI], and so on. My opinion is none of these surveys are worth a lot because people are living in denial, he contends.

The Problem With ROI

The philosophy behind ROI makes sense: No company has an unlimited technology budget, and it is reasonable to target investments that have the most positive business impact. Its also a deceptively simple calculationa numerator and a denominatorwhich makes it appealing and understandable. Knowing how to use a spreadsheet is all you need [to do the calculation], Jakubik says.

But Ben Salzmann, president and CEO of P&C insurer ACUITY, maintains this simplicity is exactly why focusing on ROI is a problem, particularly when it is used as a substitute for good management practice. When do executives ask for ROI? When they dont know what a project is about and are unwilling to take the time to learn, he says. If the ROI is big, theyll approve it. If its little, they wont. They dont know what the system theyve approved really is, what it will do, or if it will help or not. Its a cop-out.

The simplicity of ROI also is deceptive, hiding the complexity of assumptions behind that calculation. The difficulties of accurately calculating ROI and the problems with an ROI-focused approach to decision-making fall into four key areas:

1. Lack of information. First, most companies lack the information needed to project the return part of the equation accurately. Part of this is external. For example, the industry doesnt have standards in terms of what it should cost to produce an auto policy with one vehicle, says Deborah Smallwood, vice president at TowerGroup. There are some rules of thumb out there, but no industry metrics. She contrasts this shortcoming to the banking industry where there are standards for costs such as a typical credit card transaction.
Although insurers often turn to vendors to help project ROI, those vendors also may lack needed cross-industry benchmarks. Vendors have the same problem of lack of specialized resources [that insurers do], says Jakubik.

Changing market conditions complicate the process, as well, particularly for long-term projects. As you go from a hard market to a soft market, your [return] numbers can vary tremendously without much effort on your part, says Craig Lowenthal, vice president and CIO of Hartford Financial Products, a subsidiary of The Hartford.

Furthermore, part of the problem is internal. While large insurers have been able to create practices around and commit staff to ROI assessment, most carriers undertake the exercise only infrequently, assigning the task to staff who are unfamiliar with the process and who already have a full workload.
It is not an easy exercise to conduct in any company. You have to have a high-level expert or specialized resources to do it. You also need to be able to dedicate people to the process, Jakubik says.

Ellis, in his report, indicates most companies dont have the knowledge or tools needed to do ROI calculations. As a result, he says, the way most companies calculate ROI is inconsistent from proj-ect to project.

2. Lack of objectivity. Anyone who is part of a technology project has a vested interest in it, and that interest can influenceintentionally or unintentionallythe ROI projection.

Internally, project sponsors may present the best-case scenario, or those assigned to calculate ROI may feel pressure to produce a positive result. The business analysts [know] their boss thinks there is a business case, so theyd better go figure out if that is true, Ellis says.

Externally, insurers have turned to analysts, consultants, and vendors to assist in the calculation. The first two add cost, and someparticularly the thirdmay lack objectivity. Its hard to believe the calculations the vendor does, says Jakubik.

3. Omission of soft benefits. The ROI calculation stresses the hard-dollar benefits of a technology project: reassignment or reduction of staff, reduced processing time, or increased sales. It doesnt capture easily soft benefits of improved service, increased productivity, better relationships, or other valid, targeted objectives.

Nucleus Research, a provider of ROI-focused research and advisory services, estimates these benefits, or indirect returns, account for 50 percent of technology ROI. Although companies can assess these returnsNucleus itself offers a methodology to do soinsurers have a difficult enough time projecting direct benefits and often are skeptical of soft-benefit projections.
Cost is easy to determine, but benefits are where the difficulties lie, particularly when it comes to measurement, Jakubik says.
Therefore, rather than try to represent these benefits as hard dollars, most companies consider them outside the ROI process. There are projects where we dont achieve our targeted rate of return, but we still undertake them for various reasonscompetitive advantage, customer goodwill, needed upgrades, Lowenthal explains.

4. Lack of a post-mortem. Finally, a shortcoming of ROI calculations at most companies is the lack of follow-through on either the cost or the benefit side of the equation to see whether projected returns actually have been realized. Smallwood maintains very few projects are completed on time and within budget and scope. There are seldom penalties [for these failures], only rewards for completing projects, Smallwood says. Only today are insurers beginning to show an interest in assessing the validity of ROI calculations post-mortem, she reports.

Additionally, to perform this assessment, companies must create metrics or establish another system of measurement at the outset so they can see whether the benefit is in fact realized. Ellis points out this often is lacking in the project specifications. A benefit isnt a benefit unless someone realizes it and reports it, he says. Accountability for [realization] of benefits themselves is a new way of thinking for organizations.
Without a formal process to conduct a post-mortemincluding the interest of top levels of managementthe analysis is unlikely to be done. Business doesnt want to take the accountability for measuring the benefits, Jakubik says. Or, [the evaluation] happens years later when its difficult to assess or people who may have been involved in the proj-ect have moved on.

By omitting this analysis, businesses miss learning opportunities. Its a Catch-22because of lack of follow-up, which is caused by lack of interest, there is no organizational learning, and people are no wiser today than they were five years ago, Jakubik says.

A Better Way

With its shortcomings, why has ROI become a centralor even singularmeasurement for technology projects? Primarily, it can be attributed to the downturn in insurers financial fortunes, when IT budgets tightened and companies shifted their emphasis to tactical projects designed to cut costs.

When ROE [return on equity] is down, investments become very tactical, Smallwood says. Theres less investment in infrastructure, upgrading servers and databases, and even less emphasis on process improvement and governance. Infrastructure investment lends itself to evaluation by longer-term measurements, such as total cost of ownership, rather than a static ROI snapshot.

Also, management at companiesparticularly those subject to Sarbanes-Oxleyhas been under more pressure to justify and verify its financial decisions. ROI gave managers a concrete number to point to if problems later arose.

Today, Smallwood contends insurers current financial fortunes have allowed their IT departments to emerge from under paralyzing business control. Coming out of a hard market, ROE is getting back to where companies, particularly in P&C, are more comfortable [making technology investments]. Budgets are opening up, and trends are more strategic, she says.
It boils down to the leadership understanding the value of IT and how investments in IT can enable a business and be a competitive weapon. If [insurers] view IT as a cost to doing business, they are going to be ROI driven. But when you look at the leaders, they are letting strategy decide where to invest, Smallwood adds.

At Hartford Financial Products, for example, technology projects, depending upon investment level, undergo an ROI calculation that considers cost components such as software, hardware, maintenance, and personnel. While the insurer does target an internal rate of return (which it would not specify), Lowenthal stresses the ROI calculation is only one component of a larger cost-benefit analysis (CBA).

Business needs take precedence over the specific ROI, Lowenthal says. He also notes both the ROI and CBA processes have been much less formalized at Hartford Financial Products than at its parent company. He attributes this to smaller staff size that allows closer working relationships between business and IT at HFP and more fluid evaluation of the needs of business and the impact of technology. The Hartford has begun putting processes and governance in place to strengthen the use and validity of ROI calculations throughout the organization as part of its demand management transformation, he continues, which is designed to make ROI more relevant by broadening the factors that can be included in determining ROI.

Lowenthal describes two recent, related projects at Hartford Financial Products where ROI was not a driving factor in the decision. In 2001, the insurer had been looking for a way to improve how staff used the legacy WINS policy and claims administration system (from Wheatley Insurance Systems) by updating the user interface and adding a workflow system. However, September 11th interrupted planning on this project and created a more pressing business need when all the insurers paper files in its former World Trade Center offices were destroyed.

A decision was made very quickly we would recreate all these files electronically and, going forward, create an electronic copy of all paper files, Lowenthal explains, adding the decision was made without regard to ROI and with very little cost-benefit analysis. We needed to do this for disaster and recovery purposes.

The company realized the opportunity to combine this initiative with the legacy system project. It created two internally developed systems, eFile for electronic document management and VISion (Virtual Insurance System) for workflow, both deployed in early 2003. VISion is a Java-based front end that allowed HFP to retain its existing investment in its AS/400-based WINS system.

Previously, underwriters would have to do double and triple entry and process different business different ways. That took time away from servicing customers and writing new business, says Lowenthal. VISion provides a single point of data entry and an automated workflow that both directs tasks to appropriate staff members and allows those staff members to call other needed applications, such as e-mail and Word for follow-up correspondence and documentation.

Hartford Financial Products AS/400 also serves as the repository for the eFile system. Staff can save e-mail and Word documents to the system by clicking an eFile toolbar button within those applications, with VISion automatically applying indexing metadata to documents for ease of search and retrieval. The insurer invested in a pair of production scanners and scanner add-ons for all of the individual laser printers located on underwriters and claims representatives desks throughout the company to capture and index all incoming documents.

Lowenthal attributes these initiatives with helping the company handle a tripling of revenue over the past four years with virtually no increase in staff and in keeping expenses extremely low. Our goal is to offer great products and have great customer service as opposed to strictly targeting ROI, he says, but if we can grow business without hiring additional people, thats great.
Hartford Financial Products approach reflects Smallwoods recommendation for how insurers make technology investment decisions today. ROI has to be a component of the decision-making, she says, [but] you cant be purely ROI driven.
ACUITY, however, sees even less value to the ROI calculation in decision-making. You cant do an ROI on a five-year project to go paperless, Salzmann says. Only when a project is far from strategic, where the decision is dealing with some last little component that would deliver the same impact regardless of which option was chosen, will we even look at ROI.
The paperless project Salzmann refers to began with the insurers installation of IBMs Content Manager system, which has been in place at ACUITY since 1998. In 2002, the company rolled out Content Manager OnDemand, providing real-time access by staff and agents to claims and policy data via the ACUITY Web site, as well as IBMs thin-client Enterprise Information Portal, which also gave field staff Internet-based access to Content Manager. All photographs and dictation, such as for loss control surveys and claims statements, either are created as or transformed into digital files and stored in Content Manager.

Additionally, other projects have fed into this paperless initiative. The insurer receives the majority of its new-business applications electronically in both commercial and personal lines, either from its Web portal or via real-time upload from various agency management systems. It also delivers personal lines policies electronically as PDF files to agents rather than mailing print copies.

A key tool in ACUITYs decision-making process, Salzmann explains, is a strategic filtera one-page, yes/no scorecard that focuses exclusively on benefit, rather than cost. He says the goal is to look objectively at ideasregardless of where they originatedthat might otherwise be viewed with unmerited subjective optimism. Many major decisions are made by committees that represent cross-sections of the insurers business areas.

It may seem too simple to distill a business case to one page, but thats exactly what Ellis recommends. Most of the time when you get a business case, there are 10 pounds of factual information, and it is difficult to sort through exactly what youre trying to achieve. We espouse putting that business case on a single piece of paper. Represent that argument in terms of how those benefits would be achieved, and link those benefits to some end goal in the organization.

The measure of performance isnt how much you cut costs, its how well you achieve long-term goals, Salzmann says. Sometimes that means spending more than other companies when it supports our strategy of being an underwriting company and building partnerships with our agencies. We know that will ultimately improve our bottom line more than with an ROI-focused approach to technology investment.

While cost cutting isnt ACUITYs stated objective, the insurer has reduced its overall expense ratio by more than 10 points over the past five years, despite double-digit increases in its technology budget.

Ultimately, Jakubik believes, if you look beyond todays claims of stringent focus on ROI, you will find most carrierslike these insurersactually are taking a business-case approach to technology investment. Many talk about [calculating ROI], but very few do it right, he says. They compromisethey simplify the process or the calculations and [ultimately] take an intuitive approach. And theres not much wrong with running business intuitively if youre good at it.

The ROI of ROI is zero, says Ellis. Its a proxy, a number. Its whats behind the number, how its constructed, and the quality of that processthat is most relevant.