Material Insights · Understanding the Material World

Information theory: Discovery

The story

How the idea was found and came to be accepted

In 1948 Claude Shannon, working at Bell Labs, published 'A Mathematical Theory of Communication' in two parts [1, 3]. He framed communication as the problem of reproducing at one point a message chosen at another, measured information in binary digits, or bits, a word suggested by his colleague J. W. Tukey, and defined a channel's capacity [1]. His key result was that errors can be made as rare as you wish at any rate below capacity, and cannot be avoided above it [1]. It was an immediate success with communications engineers [3], but it only proved that good codes exist. Hamming published the first practical error-correcting codes in 1950 [7], Gallager devised LDPC codes by 1962 and they were forgotten for over 30 years, and turbo codes in 1993 came within about 0.7 dB of the limit [5]. Today the goal has been reached in many practical cases [5].

as of 2026-11-12

People

The people behind it, their years and what each contributed

NameYearsContribution
Claude Shannon1916 to 2001Founded the theory in 1948 and defined bits and channel capacity [1, 2, 3]
Richard Hamming1915 to 1998Colleague of Shannon at Bell Labs who published practical error-correcting codes in 1950 [5, 6, 7]

as of 2026-11-12

Milestones

The key moments, year by year

YearEvent
1948Shannon publishes 'A Mathematical Theory of Communication', appearing to give the first published use of the word bit [1, 3]
1950Hamming's 'Error Detecting and Error Correcting Codes' appears in the Bell System Technical Journal [7]
1962Gallager has developed LDPC codes and their decoder; the work is then largely forgotten for more than 30 years [5]
1993Turbo codes are introduced at a conference in Geneva and come within about 0.7 dB of the limit [5]

as of 2026-11-12

Sources

  1. A Mathematical Theory of Communication The Bell System Technical Journal (C. E. Shannon), 1948
  2. Claude Shannon Article Encyclopaedia Britannica
  3. Claude Shannon (print view of the Britannica biography) Encyclopaedia Britannica
  4. Channel Coding: The Road to Channel Capacity Proceedings of the IEEE, on arXiv (D. J. Costello Jr. and G. D. Forney Jr.), 2006
  5. Computer Pioneers - Richard Wesley Hamming IEEE Computer Society (from J. A. N. Lee, Computer Pioneers, 1995, updated)
  6. Richard Hamming ACM A.M. Turing Award (biography by Edmund F. Robertson)