Deep learning: Evolution
Timeline
The milestones from first idea to today
| Year | Milestone |
|---|---|
| 1958 | Frank Rosenblatt shows the Perceptron Mark I to the public [2] |
| 1983 | Hinton and Sejnowski invent Boltzmann machines [1] |
| 1986 | Rumelhart, Hinton and Williams publish backpropagation for multilayer networks [1] |
| 1998 | A paper reports that a cheque-reading system built on convolutional networks is in commercial use, reading several million cheques a day [7] |
| 2000 | Bengio's neural probabilistic language model introduces word embeddings, a way of representing word meaning [1] |
| 2009 | ImageNet, an image collection far larger than any before, is completed [2] |
| 2012 | Hinton's students almost halve the ImageNet error rate with a deep network trained on two Nvidia cards [1][2] |
| 2024 | Hopfield and Hinton are awarded the Nobel Prize in Physics for neural network learning [3] |
as of 2026-11-11
Lineage
The technologies it came from and the ones that came from it
Builds on- GPU Graphics processing unit AlexNet was trained in 2012 on two Nvidia graphics cards, which suit the many parallel matrix multiplications of training [2].
- SPC Stored-program computer A neural network is a program whose behaviour is set by learned numbers rather than hand-written rules, run on an ordinary computer.
- LLM Large language models Large language models are deep networks; Bengio's neural language model of 2000, which introduced word embeddings, was an early step [1].
as of 2026-11-11
Sources
- Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award Association for Computing Machinery, news release distributed on EurekAlert!, 2019
- CHM Releases AlexNet Source Code Computer History Museum (Hansen Hsu), 2025
- Press release: The Nobel Prize in Physics 2024 The Royal Swedish Academy of Sciences, NobelPrize.org, 2024
- Gradient-based Learning Applied to Document Recognition Proceedings of the IEEE (Yann LeCun, Leon Bottou, Yoshua Bengio and Patrick Haffner), article page, 1998