Not all system failures are created equal; there certainly are degrees of failure. This article describes the types and consequences of defines four major types of IT project failures. Note that one failure may not become apparent until months after a new system has been activated. Each case study is classified in terms of the following failure scale:

• The Unmitigated Disaster
• The Big Failure
• The Mild Failure
• The Forthcoming Failure

The Unmitigated Disaster
The most egregious failure occurs when an organization spends millions of dollars implementing a system and misses deadlines repeatedly. It ultimately junks the new system for a different one altogether or reverts to the legacy system. Relationships between consultancies and clients are often severed. Lawsuits in such cases are not completely out of the question. Fortunately, these abominations are atypical.

The Big Failure
These types of failures are less severe but more common. Perhaps an organization initially budgets $2M and one year on an implementation and ultimately spends $4M over the course of three years, getting much less functionality than expected in the process.

The Mild Failure
Very often, a system failure is so mild that one can hesitate to even call it a failure, especially relative to the two types just mentioned. By comparison, these are rousing successes! For the sake of consistency, however, this article uses the term “failure.” An example of the Mild Failure is the company that initially budgets $2M and a year on an implementation and ultimately spends $2.2M over the course of fifteen months, getting slightly less functionality than expected in the process.

The Forthcoming Failure
Sometimes a system failure is not immediately apparent. At first, this notion may seem perplexing. If an organization has met its goals with respect to both its budget and deadline, then how can it consider the system a failure?
Budget and deadline are only two criteria for a system failure, as the statistics at the beginning of the chapter illustrate. The answer to this question lies within the organization’s data, documentation, processing, and people. Examples of latent failures include the following:

• The implementation team has made a key mistake that will come back to haunt the organization down the road. 
• End-users may not completely understand the system and, as a result, make significant errors or revert to “old ways,” negating one of the major benefits of the new system.
• The organization is vulnerable to employee attrition on two fronts:
  • End-user documentation is deficient and, if key staff members leave, their replacements will need significant time and/or training to do their jobs.
  • Knowledge is not dispersed; only a few employees understand the system in sufficient breadth and depth.

In other words, these are failures waiting to happen. Organizations are not prepared for shocks to their systems.

A Prime Example of the Forthcoming Failure
I am reminded of an organization—Oates Healthcare—that activated its ERP in 2003 with a fundamental but unknown problem with the way in which it calculated employee overtime. No one identified this issue during setup or testing. Only when an ex-employee filed a lawsuit did the problem come to light, five years after Oates had gone live.

For Oates, fixing the problem in the system involved two things, one simple and one very difficult. The first merely entailed changing some flags in the system, allowing it to begin calculating overtime correctly from that point forward. However, checking those flags did not retroactively go back and recalculate overtime for all employees paid incorrectly over the past five years. A breakdown of those errant employee records is presented in Table 1.2:

Breakdown of Payroll Records Requiring Analysis at Oates Healthcare
Employees paid per year 6,500
Average types of pay per week per employee 6
Weeks per year 52
Checks per year 338,000
Payroll records per year 2,028,000
Years of data requiring analysis 5
Total records (over a five year period) 10,140,000

The enormity of this task—recalculating employee overtime—was beyond the time and skill of Oates’ existing end-users (arguably a failure in itself). Even if an internal super-user knew how to do this on over 10,000,000 records, he or she would not have been able to do it. For ad hoc analyses, Oates provided its end-users only with Microsoft Excel, a very valuable tool but one not nearly robust enough to handle a task of this magnitude. As a result, Oates hired Bishop Consultants to perform this task at considerable expense.

Had Oates’ end-users properly tested the system prior to going live, it may have avoided the lawsuit. To be sure, it would have not have had to spend the time, internal resources, and funds on Bishop to fix the problem. Bishop recalculated employee overtime pay but Oates’ end-users were not available during the remediation project. This prohibited Bishop consultants from transferring any knowledge during the error-resolution process.
Ironically, Oates did not learn from its mistakes. Despite the recommendation from Bishop, Oates did not seriously consider adding a more powerful reporting tool for end-users to conduct similar kinds of analyses (e.g., Crystal Reports, Microsoft Access, or Business Objects). Oates also failed to actively recruit more technical end-users to use these very tools, should it one day have decided to purchase them. If Oates encounters a similar problem in the future, then it will be at the mercy of external consultants such as Bishop once again.

Consequences of a Typical System Failure
The consequences for a failed implementation go beyond mere dollars and cents. Let’s return to the P&P example. Bonham Software may forever have a tarnished reputation within the organization among both end-users and employees who actually do not even use the system on a regular basis. Bonham may always be known as “that system that screwed up payroll.” Data could be lost or altered in such a way that it will be impossible to retrieve. Due to lack of training or documentation, employees’ jobs may actually become more difficult than they were with P&P’s legacy system.

For many software vendors, the IT project failures are disasters. They are often left with tarnished reputations resulting from highly publicized failures. They may have difficulty collecting the hundreds of thousands in accounts receivable and lose key consultants. Clients may also refuse to provide a reference for its new partner.

Taken from "Why New Systems Fail" by Phil Simon