# Deep learning: Evolution

Source page: https://www.materialinsights.org/technologies/deep_neural_networks/tech-evolution/

#### Timeline

The milestones from first idea to today

| Year | Milestone |
|---|---|
| 1958 | Frank Rosenblatt shows the Perceptron Mark I to the public ^[2] |
| 1983 | Hinton and Sejnowski invent Boltzmann machines ^[1] |
| 1986 | Rumelhart, Hinton and Williams publish backpropagation for multilayer networks ^[1] |
| 1998 | A paper reports that a cheque-reading system built on convolutional networks is in commercial use, reading several million cheques a day ^[7] |
| 2000 | Bengio's neural probabilistic language model introduces word embeddings, a way of representing word meaning ^[1] |
| 2009 | ImageNet, an image collection far larger than any before, is completed ^[2] |
| 2012 | Hinton's students almost halve the ImageNet error rate with a deep network trained on two Nvidia cards ^[1]^[2] |
| 2024 | Hopfield and Hinton are awarded the Nobel Prize in Physics for neural network learning ^[3] |

as of 2026-11-11

#### Lineage

The technologies it came from and the ones that came from it
 Builds on

- [GPU Graphics processing unit](https://www.materialinsights.org/technologies/graphics_processing_unit/) AlexNet was trained in 2012 on two Nvidia graphics cards, which suit the many parallel matrix multiplications of training ^[2].
- [SPC Stored-program computer](https://www.materialinsights.org/technologies/stored_program_computer/) A neural network is a program whose behaviour is set by learned numbers rather than hand-written rules, run on an ordinary computer.
 Enables

- [LLM Large language models](https://www.materialinsights.org/technologies/large_language_models/) Large language models are deep networks; Bengio's neural language model of 2000, which introduced word embeddings, was an early step ^[1].

as of 2026-11-11

#### Sources

1. [Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award](https://www.eurekalert.org/news-releases/508417) Association for Computing Machinery, news release distributed on EurekAlert!, 2019
2. [CHM Releases AlexNet Source Code](https://computerhistory.org/blog/chm-releases-alexnet-source-code/) Computer History Museum (Hansen Hsu), 2025
3. [Press release: The Nobel Prize in Physics 2024](https://www.nobelprize.org/prizes/physics/2024/press-release/) The Royal Swedish Academy of Sciences, NobelPrize.org, 2024
4. [Gradient-based Learning Applied to Document Recognition](https://proceedingsoftheieee.ieee.org/gradient-based-learning-applied-to-document-recognition/) Proceedings of the IEEE (Yann LeCun, Leon Bottou, Yoshua Bengio and Patrick Haffner), article page, 1998

## Other sections

- [Summary](https://www.materialinsights.org/technologies/deep_neural_networks/index.md)
- [Overview](https://www.materialinsights.org/technologies/deep_neural_networks/tech-overview/index.md)
- [Science](https://www.materialinsights.org/technologies/deep_neural_networks/tech-science/index.md)
- [Energy footprint](https://www.materialinsights.org/technologies/deep_neural_networks/tech-energy-footprint/index.md)
- [Industries](https://www.materialinsights.org/technologies/deep_neural_networks/tech-industries/index.md)
