Information theory: Discovery
The story
How the idea was found and came to be accepted
In 1948 Claude Shannon, working at Bell Labs, published 'A Mathematical Theory of Communication' in two parts [1, 3]. He framed communication as the problem of reproducing at one point a message chosen at another, measured information in binary digits, or bits, a word suggested by his colleague J. W. Tukey, and defined a channel's capacity [1]. His key result was that errors can be made as rare as you wish at any rate below capacity, and cannot be avoided above it [1]. It was an immediate success with communications engineers [3], but it only proved that good codes exist. Hamming published the first practical error-correcting codes in 1950 [7], Gallager devised LDPC codes by 1962 and they were forgotten for over 30 years, and turbo codes in 1993 came within about 0.7 dB of the limit [5]. Today the goal has been reached in many practical cases [5].
as of 2026-11-12
People
The people behind it, their years and what each contributed
| Name | Years | Contribution |
|---|---|---|
| Claude Shannon | 1916 to 2001 | Founded the theory in 1948 and defined bits and channel capacity [1, 2, 3] |
| Richard Hamming | 1915 to 1998 | Colleague of Shannon at Bell Labs who published practical error-correcting codes in 1950 [5, 6, 7] |
as of 2026-11-12
Milestones
The key moments, year by year
| Year | Event |
|---|---|
| 1948 | Shannon publishes 'A Mathematical Theory of Communication', appearing to give the first published use of the word bit [1, 3] |
| 1950 | Hamming's 'Error Detecting and Error Correcting Codes' appears in the Bell System Technical Journal [7] |
| 1962 | Gallager has developed LDPC codes and their decoder; the work is then largely forgotten for more than 30 years [5] |
| 1993 | Turbo codes are introduced at a conference in Geneva and come within about 0.7 dB of the limit [5] |
as of 2026-11-12
Sources
- A Mathematical Theory of Communication The Bell System Technical Journal (C. E. Shannon), 1948
- Claude Shannon Article Encyclopaedia Britannica
- Claude Shannon (print view of the Britannica biography) Encyclopaedia Britannica
- Channel Coding: The Road to Channel Capacity Proceedings of the IEEE, on arXiv (D. J. Costello Jr. and G. D. Forney Jr.), 2006
- Computer Pioneers - Richard Wesley Hamming IEEE Computer Society (from J. A. N. Lee, Computer Pioneers, 1995, updated)
- Richard Hamming ACM A.M. Turing Award (biography by Edmund F. Robertson)