Material Insights · Understanding the Material World

Deep learning: Evolution

Timeline

The milestones from first idea to today

YearMilestone
1958Frank Rosenblatt shows the Perceptron Mark I to the public [2]
1983Hinton and Sejnowski invent Boltzmann machines [1]
1986Rumelhart, Hinton and Williams publish backpropagation for multilayer networks [1]
1998A paper reports that a cheque-reading system built on convolutional networks is in commercial use, reading several million cheques a day [7]
2000Bengio's neural probabilistic language model introduces word embeddings, a way of representing word meaning [1]
2009ImageNet, an image collection far larger than any before, is completed [2]
2012Hinton's students almost halve the ImageNet error rate with a deep network trained on two Nvidia cards [1][2]
2024Hopfield and Hinton are awarded the Nobel Prize in Physics for neural network learning [3]

as of 2026-11-11

Lineage

The technologies it came from and the ones that came from it

Builds on
  • GPU Graphics processing unit AlexNet was trained in 2012 on two Nvidia graphics cards, which suit the many parallel matrix multiplications of training [2].
  • SPC Stored-program computer A neural network is a program whose behaviour is set by learned numbers rather than hand-written rules, run on an ordinary computer.
Enables
  • LLM Large language models Large language models are deep networks; Bengio's neural language model of 2000, which introduced word embeddings, was an early step [1].

as of 2026-11-11

Sources

  1. Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award Association for Computing Machinery, news release distributed on EurekAlert!, 2019
  2. CHM Releases AlexNet Source Code Computer History Museum (Hansen Hsu), 2025
  3. Press release: The Nobel Prize in Physics 2024 The Royal Swedish Academy of Sciences, NobelPrize.org, 2024
  4. Gradient-based Learning Applied to Document Recognition Proceedings of the IEEE (Yann LeCun, Leon Bottou, Yoshua Bengio and Patrick Haffner), article page, 1998