Deep learning: Science
Principles
The science it rests on, with why each principle matters to it
Relies on- Artificial neural networks Deep networks are built from layers of artificial neurons, first made to learn in Rosenblatt's perceptron of 1958 [2].
- Backpropagation and gradient learning Backpropagation, set out in a 1986 paper by Rumelhart, Hinton and Williams, is how every deep network is trained [1].
- Probability and statistics Networks such as the Boltzmann machine learn the probabilities of patterns in their data [1][3].
as of 2026-11-11
Discoveries
The discoveries that made it possible: when, by whom and what was found
| Year | Who | What |
|---|---|---|
| 1958 | Frank Rosenblatt, Cornell University | The perceptron, a machine that learned to classify inputs, shown to the public [2] |
| 1983 | Geoffrey Hinton and Terrence Sejnowski | Boltzmann machines, networks that learn the patterns in their training data and can classify or create new examples [1][3] |
| 1986 | David Rumelhart, Geoffrey Hinton and Ronald Williams | Backpropagation for training networks with hidden layers [1] |
as of 2026-11-11
The science explained
The science behind it explained for a curious reader
Imagine a long line of people passing a message, each one allowed to turn it up or down a little before passing it on. The last person says what the picture shows. If the answer is wrong, a note goes back down the line telling each person how much their own adjustment added to the mistake, and each changes it a little. That backward note is backpropagation [1]. After millions of pictures, the people near the start of the line have learned to pick out edges, those in the middle shapes, and those at the end whole objects. Nobody told them what to look for; the layers found it in the data. Hopfield and Hinton won the 2024 Nobel Prize in Physics for the foundations of this kind of learning [3].
as of 2026-11-11
Sources
- Fathers of the Deep Learning Revolution Receive ACM A.M. Turing Award Association for Computing Machinery, news release distributed on EurekAlert!, 2019
- CHM Releases AlexNet Source Code Computer History Museum (Hansen Hsu), 2025
- Press release: The Nobel Prize in Physics 2024 The Royal Swedish Academy of Sciences, NobelPrize.org, 2024