Material Insights · Understanding the Material World

Artificial neural networks

Networks of simple threshold units can compute and learn from examples.

Mathematics and information · Machine learning · No. 125

  • Electrical and chemical age
  • Science
Mathematics and information · The science behind technology

A 1943 model that only added up inputs and fired led to a 1958 machine that taught itself to sort cards.

Chart: Illustration: Artificial neural networks

Where it comes from

Chart: Map of where it comes from

Warren McCulloch and Walter Pitts published their logical model of the neuron in 1943. Frank Rosenblatt turned it into a learning machine, the perceptron, which the US Navy unveiled in July 1958.

Three things to know
  1. McCulloch and Pitts used the all-or-none firing of nerve cells to treat neural events with propositional logic.
  2. In July 1958 the US Office of Naval Research unveiled Rosenblatt's perceptron, which taught itself to tell cards marked on the left from cards marked on the right.
  3. A single layer of such units cannot solve the XOR problem, and the limit is blamed for the AI winter of the 1970s.
The journey
  1. 1 Nerve cellFires all or nothing when its inputs add up enough
  2. 2 ModelMcCulloch and Pitts treat neurons with logic, 1943
  3. 3 LearningRosenblatt's perceptron adjusts its connections, 1958
  4. 4 SetbackMinsky and Papert show the limits of one layer, 1969
  5. 5 RevivalMany-layer networks trained by backpropagation, 1986
  6. 6 TodayVision, speech, robots and language models
Go deeper pick a tab to open it in full