Case 3: The 35% Problem — Innovare AI Analysis

Module 3 · Video Walkthrough

Module 3 · Quiz Cheat Sheet

As you read — hold this question

If a number appears in thousands of business articles but no one can trace it to an original methodology — what should you do with it in a business case?

35%

of Amazon's revenue attributed to its recommendation engine. First published 2012. Never independently verified. Possibly 11%. Possibly not AI by today's definition.

The 35% figure has travelled through more than a decade of AI presentations, business cases, and academic papers. It traces to a McKinsey article from 2012 — citing Amazon at a point when the recommendation system being described was collaborative filtering, not machine learning as the term is used today. A University of Florida replication study found 11%. Amazon has never confirmed either number. The technology described is from an era when "AI" and "machine learning" were not yet the same sentence. This is the benchmark problem in AI: unverified numbers move faster than corrections, and business cases built on them inherit the error.

Tracing the claim — source by source

Quiz: Source evaluation

What we know, what we don't, and what that means for how we use it

The claim

"35% of Amazon's revenue comes from its recommendation engine." Appears across consulting reports, AI vendor marketing, academic textbooks, and MBA case studies.

The source

McKinsey & Company (2013). "How retailers can keep up with consumers." Cited as an observation, not a primary study. McKinsey did not conduct the revenue analysis — they referenced Amazon's own statements from an executive presentation. No methodology disclosed.

The tech

The system being described in 2012 was collaborative filtering — a statistical method that groups users by purchase patterns. Not deep learning. Not a neural network. The question of whether this constitutes "AI" depends on the definition in use. By 2026 standards, it's more accurately called a recommendation algorithm. The technology has changed. The label hasn't.

Replication

A University of Florida study attempting to replicate the figure found approximately 11% revenue attribution. Different methodology, different time period, different interpretation of what "from recommendations" means. Neither 35% nor 11% has been confirmed by Amazon.

Current status

Amazon has not published revenue attribution data for its recommendation system at any point. The figure continues to circulate. Best use: cite with full provenance and known uncertainty range (11%–35%), not as a standalone fact.

Why unverified numbers travel — and why this one specifically

The benchmark problem in AI measurement

The 35% figure travels for the same reasons most unverified AI benchmarks travel:

It's a good number. 35% is large enough to be significant, round enough to be memorable, and attributable to a credible enough company that it's hard to argue with in a meeting.
Corrections don't travel as fast. The original claim is cited in hundreds of places. The correction (UF study, 11%) is cited in almost none of them.
Vendors have incentives to repeat it. AI recommendation system vendors benefit from a world where the benchmark is 35%, not 11%.
Nobody checks the original source. Most citations of the McKinsey article are secondary or tertiary — people citing people who cited people who cited McKinsey. The original has no primary data.

The deeper question — is this even the right benchmark?

Even if we accepted 35% at face value, it's a revenue attribution figure — not an ROI figure, not a profit figure, not a marginal-contribution figure. Revenue attributed to a recommendation engine is not the same as revenue that wouldn't have happened without it. Amazon has recommendations everywhere — the baseline without them is unmeasurable. The 35% claim answers a question that nobody actually asked, with a methodology nobody actually reviewed, about a technology that may not qualify as AI by today's definition. That's three separate problems before the accuracy question even starts.

Applied — a framework for evaluating AI claims

Four questions to ask before citing a number in a business case

Question	What you're checking	Amazon 35% — the answer
Who produced the original data?	Is this primary research or a secondary citation? Does the producer have a conflict of interest?	McKinsey cited an Amazon executive statement. No primary study. Consulting firms often reference client data without methodology disclosure.
When was it measured?	Is the technology or market context still relevant? Has the system changed significantly since then?	2012. Pre-deep learning at scale. The technology described is collaborative filtering. Modern Amazon recommendations are substantially different systems.
Has it been replicated?	Has any independent study attempted to verify the finding? What did they find?	One known replication study (University of Florida) found 11%. No other independent verifications. Amazon has not confirmed.
Is the metric the right one?	Does the metric actually measure what the claim says it measures? Revenue attribution ≠ ROI ≠ profit.	Revenue attribution measures click-through, not counterfactual. It doesn't tell you what would have been purchased without the recommendation. Not comparable to financial ROI.

Take this away

In AI measurement, provenance matters as much as the number. A claim without a traceable methodology isn't evidence — it's a talking point. When you use it in a business case, you inherit the uncertainty. When it gets challenged, you're the one explaining why you didn't check.

← Case 2: The Adoption Curve

Case 4: Porter's Three Strategies →

Quick recall — without looking back

Test yourself on this case

Question 1 of 3

Where does the 35% Amazon recommendation figure come from — and what are the three specific problems with using it uncritically?

Source: A 2013 McKinsey article that cited an Amazon executive statement. No primary study, no disclosed methodology. Three problems: (1) Provenance — it's a secondary citation of an undisclosed data point with no independent verification. (2) Currency — the technology described in 2012 was collaborative filtering, not modern machine learning; the system has changed significantly since. (3) Metric appropriateness — revenue attribution is not the same as ROI; it doesn't measure what would have been purchased without the recommendation (the counterfactual is unmeasurable).

Question 2 of 3

What did the University of Florida replication study find — and why doesn't the correction receive the same attention as the original claim?

The UF study found approximately 11% revenue attribution — less than a third of the McKinsey-cited figure. The correction doesn't travel for structural reasons: (1) the original claim is embedded in hundreds of articles, textbooks, and consulting decks, while the correction exists in one academic study; (2) vendors who benefit from the higher number have incentives to repeat the original; (3) most citations of "35%" are tertiary — people citing people who cited McKinsey — so the correction never reaches them. This is the standard pattern for unverified AI benchmarks: the claim scales, the correction doesn't.

Question 3 of 3

Name the four questions you should ask before citing an AI performance figure in a business case.

(1) Who produced the original data? — Is this primary research or a secondary citation? Does the source have a conflict of interest? (2) When was it measured? — Is the technology or market context still current? Has the system changed significantly since then? (3) Has it been replicated? — Has any independent study verified the finding? What did they find? (4) Is the metric the right one? — Does the metric actually measure what the claim says it measures? Revenue attribution, ROI, profit contribution, and productivity gain are different things that often get conflated in AI performance discussions.

Sources

McKinsey (2013): Bughin, J. (2013). How retailers can keep up with consumers. McKinsey Quarterly. (Primary carrier of the 35% figure.)
Amazon case: BUSN9049 Module 3 Case Study 3.5 — The Business Value of AI: Amazon's Recommendation System. Flinders University, 2026.
Module transcript: BUSN9049 Module 3 Part 3 — Understanding Return on Investment from AI. Flinders University, 2026.
Note: The University of Florida replication study (11% figure) is referenced in course materials; the original citation was not independently confirmed during this review. Treat the range 11%–35% as the honest representation of available evidence.