Case 5: The Five Factors — Innovare AI Analysis

Module 3 · Video Walkthrough

Module 3 · Quiz Cheat Sheet

As you read — hold this question

If the technology itself isn't usually the reason an AI project fails — what is?

5 factors

that determine whether the implementation dip becomes a recovery or a permanent failure. None of them are about which model you chose.

GE spent billions on AI and retreated. IBM Watson Health was sold. Countless enterprise AI projects stall in proof of concept and never reach production. In almost none of these cases was the root cause the AI technology itself. The technology was real and often worked as advertised. The failures happened in the five areas that surround the model: the upfront costs organisations didn't plan for, the data that wasn't clean enough to train on, the existing systems that wouldn't integrate, the talent that wasn't available or wasn't empowered, and the maintenance costs that nobody budgeted for. These five factors predict implementation outcomes better than the model selection decision does.

The five factors — what they mean in practice

Quiz: Implementation factors

Five factors that determine whether the dip becomes a recovery

Upfront Investment

Technology acquisition, development, data preparation, training

What it means

Initial costs include technology acquisition, system development, data preparation, and training — both of the AI model and of the people using it. These costs are typically underestimated at the business case stage, and the gap between projection and reality is what causes leadership to lose confidence in the J-curve before it curves back up.

Red flag

The business case shows a "software licence + implementation fee" and not much else. Data preparation, integration work, and user training are not line items — or they're estimated at 10–20% of tech cost when they're often 50–100% of it. Projects that underestimate upfront investment run out of runway before they reach the value phase.

Data Quality

Relevance, cleanliness, volume, availability, governance

What it means

The quality and relevance of training data directly determines model performance. Poor-quality data produces a model that makes confidently wrong predictions — which is worse than no model because it erodes trust faster. Data problems are also notoriously slow to fix: cleaning and governing enterprise data often takes months longer than planned.

Red flag

"We have lots of data" without a data quality assessment. Volume is not quality — a million rows of inconsistently labelled, duplicate-heavy, historically biased data will train a worse model than fifty thousand rows of clean, representative data. The data audit should happen before the model selection, not after.

System Integration

Compatibility, migration, continuity, interoperability

What it means

AI is not a standalone tool — it has to connect to existing systems, data sources, and workflows. The integration work is where most implementation timelines slip: legacy system incompatibility, API limitations, data migration complexity, and continuity requirements (the old system can't go dark while the new one is being connected) all add time and cost that weren't in the plan.

Red flag

Integration described as a "technical matter" rather than a project risk. Legacy system age, API availability, and data migration complexity should be assessed before commitment, not discovered during implementation. The "plug it in" assumption has caused more AI project failures than bad models have.

Talent

AI specialists, domain experts, data engineers, end users

What it means

Successful AI implementation requires a team that doesn't exist in most organisations: AI specialists who understand the models, domain experts who understand the business context the models need to serve, and data engineers who can manage and prepare the data pipeline. Each of these is in high demand and short supply. And end users — often overlooked — determine whether the model gets used at all.

Red flag

Assuming the vendor will provide the domain expertise. Vendors know the model — they don't know your processes, your edge cases, or your regulatory requirements. Domain expertise has to come from inside. Projects that rely entirely on vendor implementation teams tend to end up with a model that works for the vendor's reference case, not yours.

Maintenance

Ongoing costs, model drift, retraining, evaluation, support

What it means

AI systems are not set-and-forget. Models drift as the world changes — a fraud detection model trained on 2023 fraud patterns will underperform against 2026 patterns. Maintenance costs include ongoing monitoring, periodic retraining, performance evaluation, system upgrades, and technical support. These costs are often absent from the initial business case.

Red flag

Business case shows year 1 ROI and nothing else. AI systems require continuous investment to maintain performance. A model that was performing at 92% accuracy at launch may be at 78% eighteen months later if not maintained — and nobody will notice until something goes wrong.

The pattern across AI failures

What the biggest AI implementation failures had in common

Looking at the most-cited AI implementation failures — GE Digital / Predix, IBM Watson Health, high-profile retail and logistics deployments — a consistent pattern emerges:

The technology worked. GE's Predix platform could process industrial sensor data. Watson could read medical literature. The technology did what the vendor said it would do.
The integration didn't. Predix couldn't connect to the diversity of industrial equipment GE's customers actually ran. Watson's medical knowledge didn't translate to the specific protocols, liability requirements, and workflow constraints of real clinical environments.
The domain expertise was underestimated. Both GE and IBM underestimated how much domain-specific knowledge was required to make the models useful in context.
The maintenance wasn't funded. Organisations that bought in expected the initial implementation to be the cost. The ongoing cost of keeping models current and connected wasn't planned for.

The Real Lesson from GE Digital

GE didn't fail because Predix was bad AI. GE failed because they built a platform and then tried to find problems it could solve — instead of starting with the problem and building to solve it. That's the problem-first vs. AI-first error. The five factors don't save you from that mistake — they're what determines whether a problem-first project can actually execute once the right problem is identified.

Take this away

The AI model is rarely the reason a project fails. The five factors — upfront investment, data quality, system integration, talent, and maintenance — are where the execution gap opens. A good model deployed against all five of these problems will underperform a mediocre model deployed in a well-prepared environment.

← Case 4: Porter's Three Strategies

Case 6: Human in the Loop →

Quick recall — without looking back

Test yourself on this case

Question 1 of 3

Name the five factors that affect AI implementation ROI — and for each one, give the most common red flag that indicates it's being underestimated.

(1) Upfront Investment — Red flag: business case only shows licence + implementation fee; data prep, integration, and training are missing or underestimated. (2) Data Quality — Red flag: "we have lots of data" without a quality audit; volume is confused with quality. (3) System Integration — Red flag: integration described as a "technical matter" rather than a project risk; legacy system compatibility not assessed before commitment. (4) Talent — Red flag: assuming the vendor provides domain expertise; no internal domain expert on the implementation team. (5) Maintenance — Red flag: business case shows year 1 ROI only; no ongoing budget for model monitoring, retraining, or updates.

Question 2 of 3

What is model drift — and why does it mean that ongoing maintenance costs need to be in the original business case?

Model drift: the degradation of model performance over time as the real-world patterns the model was trained on change. A fraud detection model trained on 2023 patterns will underperform in 2026 because fraud tactics evolve. A demand forecasting model trained pre-pandemic will underperform in a post-pandemic supply chain. Drift is not a bug — it's an inherent property of statistical models in changing environments. This means maintenance (periodic retraining, performance monitoring, data updates) is not optional — it's the cost of keeping the model useful. If maintenance isn't in the original business case, the organisation will discover it when something goes wrong, at which point the remediation cost is higher and the trust damage is already done.

Question 3 of 3

The GE Digital / Predix case is described as "the canonical AI implementation failure." What specifically went wrong — and which of the five factors does it illustrate?

What went wrong: GE built Predix as a general-purpose industrial IoT/AI platform and then tried to find customers whose problems it could solve — the reverse of problem-first AI. When they reached deployment, Predix couldn't connect to the diversity of industrial equipment actual customers ran (System Integration), the domain expertise required to make the models work in specific industrial contexts was massively underestimated (Talent), and the maintenance and evolution costs of a platform serving multiple industries were not planned for (Maintenance). GE also underestimated the Upfront Investment required to build out the platform to the level of reliability industrial customers required, and the Data Quality of the heterogeneous sensor data from legacy equipment created training challenges. The GE case illustrates all five factors — but the root cause was strategic: AI-first rather than problem-first.

Sources

Module transcript: BUSN9049 Module 3 Part 3 — Understanding Return on Investment from AI. Flinders University, 2026.
GE Digital: Lohr, S. (2018). G.E. Makes a Sharp Retreat on Digital Services. New York Times, October 7, 2018.
Watson Health: Ross, C. & Swetlitz, I. (2017). IBM's Watson supercomputer recommended 'unsafe and incorrect' cancer treatments. STAT News, July 25, 2017.
Davenport: Davenport, T.H. & Ronanki, R. (2018). Artificial intelligence for the real world. Harvard Business Review, 96(1), 108–116.