Before 2012, machines needed humans to describe what features to look for. After 2012, they learned the features themselves. One architecture change, one dataset, one competition — and the previous 60 years of AI became historical context.
July 2026 · Case 2 of 6
As you read — hold this question
Before 2012, AI performance was bounded by the human knowledge ceiling. What replaced that ceiling — and what replaced it with?
700+
AI medical imaging tools approved by the FDA. Most of them impossible before 2012. All of them built on deep learning. Feature learning is not just a technical achievement — it is the reason most modern AI products exist.
Google’s retinal scan AI detects signs of diabetic blindness from a photograph with higher sensitivity than most ophthalmologists. It was not programmed with rules for what diabetic retinopathy looks like. It was trained on 128,000 labelled retinal photographs and learned to see it. That is what deep learning changed: machines stopped needing humans to define the features. They learned to find them.
The Taxonomy — where deep learning sits
Quiz: Deep Learning
AI ⊃ Machine Learning ⊃ Deep Learning — what each boundary means
When media says “AI” they usually mean deep learning. When they say “machine learning” they often mean both. The nested hierarchy is not just academic tidiness — each boundary marks a different capability ceiling.
AI — The outer ring
Any technique that makes a machine do something we’d call intelligent if a human did it. Includes rule-based systems, expert systems, search algorithms, and everything below. Deep Blue was AI. Chess rules encoded by humans. No learning.
ML — The middle ring
The machine improves its performance on a task through experience. No hand-coded rules — it learns from data. Includes decision trees, linear regression, SVMs, and deep learning. Spam filters are ML. Trained on millions of labelled emails. No hard-coded spam rules.
DL — The inner ring
ML using multi-layer neural networks that learn hierarchical feature representations from raw data — no human feature engineering. Requires large datasets and GPU-scale compute. GPT-4, image recognition, AlphaGo, medical imaging — all deep learning.
Traditional ML requires feature engineering — a human expert decides what signals to feed the model. Deep learning learns the features from raw data. This is why DL works on images, audio, and text — domains where humans cannot fully articulate what the relevant features are. It is also why DL requires far more data and compute than traditional ML.
The breakthrough — ImageNet 2012
What AlexNet changed — and why the date matters
The ImageNet Large Scale Visual Recognition Challenge asked models to classify 1.2 million images into 1,000 categories. In 2011, the winning model’s error rate was 25.7%. In 2012, AlexNet — a deep convolutional neural network trained on GPUs — achieved 15.3%. Everything changed.
Before 2012 — feature engineering
Human experts defined which image features mattered (edges, corners, textures)
Models applied those hand-crafted feature detectors to raw pixels
Performance was bounded by what humans could articulate about what makes a cat look like a cat
Error rates plateaued around 25% — the human knowledge ceiling
After 2012 — feature learning
Deep networks learned their own feature detectors from millions of labelled images
Early layers detect edges. Middle layers detect shapes. Deep layers detect objects.
Performance bounded only by data volume and compute — both of which scale
By 2015: 3.6% error rate — superhuman. By 2017: the competition ended.
Applied — AI in medical imaging
700+ FDA-approved tools: what feature learning looks like in a regulated industry
Medical imaging is the clearest applied case for deep learning. Radiologists learn to detect tumours, lesions, and anomalies by reviewing thousands of scans over years of training. Deep learning does the same thing — but from 100,000+ labelled scans, at scale, in seconds.
Capability
Performance
Diabetic retinopathy detection (Google, 2016) — from retinal photographs
Outperforms GPs
Breast cancer screening (MIT CSAIL, 2020) — 5-year risk prediction from mammogram
Equal to radiologist
Lung nodule detection (FDA-approved, 2023) — reduces radiologist reading time by 30%
Deployed at scale
Novel pathology detection — conditions not in training data
Fails — no transfer
Clinical reasoning — integrating imaging with patient history and judgment
Fails — not attempted
The Principle — deep learning learns within its training distribution
Medical imaging AI outperforms humans on tasks it was trained for — and fails completely on tasks it wasn’t. This is not a bug. It is the definition of supervised deep learning. The model learned the features in its training data. Anything outside that distribution is unknown territory. This is why 700+ tools are approved for specific, narrow tasks — not for general radiology.
Take this away
Deep learning learns within its training distribution. Medical imaging AI outperforms humans on tasks it was trained for — and fails completely on tasks it wasn’t. The 700+ FDA-approved tools are approved for specific, narrow tasks because that is the only kind of task supervised deep learning can do reliably.
What is the AI ⊃ ML ⊃ DL nested hierarchy — and what does each boundary mark?
AI: any technique making a machine do something intelligent — includes rule-based systems, expert systems, and ML. ML: the machine improves through experience, no hand-coded rules. Deep Learning: ML using multi-layer neural networks that learn hierarchical feature representations from raw data — no human feature engineering. Each inner boundary marks a higher capability ceiling and greater data/compute requirements.
Question 2 of 3
What specifically changed in 2012 — and why does the date matter so much?
AlexNet — a deep convolutional neural network trained on GPUs — achieved 15.3% error on ImageNet, down from 25.7% the year before. Before 2012: humans defined which features mattered and performance was bounded by what humans could articulate (the human knowledge ceiling). After 2012: networks learned their own feature detectors from millions of images. Error rates fell to 3.6% by 2015 (superhuman) and the competition ended in 2017. The ceiling was replaced by data and compute — both of which scale.
Question 3 of 3
Medical imaging AI outperforms specialists on specific tasks but fails on others. Which does it pass — and what does the failure pattern tell you?
Passes: diabetic retinopathy detection (outperforms GPs), breast cancer 5-year risk prediction (equal to radiologist), lung nodule detection (reduces reading time 30%). Fails: novel pathology detection (conditions not in training data — no transfer) and clinical reasoning (integrating imaging with history and judgment — not attempted). The pattern reveals that deep learning learns within its training distribution. The 700+ FDA-approved tools are approved for specific, narrow tasks — not general radiology.
Sources
AlexNet
Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. NeurIPS 2012.
Google retina
Gulshan, V. et al. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy. JAMA, 316(22), 2402–2410.
FDA approvals
US FDA (2024). Artificial intelligence and machine learning enabled medical devices. fda.gov.
AlphaGo
Silver, D. et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529, 484–489.