Information theory
Information can be measured in bits, and every channel has a maximum rate for error-free messages.
Mathematics and information · Information theory · No. 126
- Electronic age
- Science
Claude Shannon showed in 1948 that every noisy channel has a top speed for error-free messages, and that clever coding can come close to it.
Chart: Illustration: Information theory
Where it comes from
Chart: Map of where it comes from
Shannon, a mathematician at Bell Labs, published his theory in two parts in July and October 1948. It began with the problem of reproducing at one point a message chosen at another.
Three things to know
- The 1948 paper appears to contain the first published use of the word bit.
- Shannon's rule says a message can be sent with as few errors as you like at any rate below capacity, but not above it.
- Gallager invented low-density parity-check codes by 1962, and they were largely forgotten for more than 30 years.
The journey
- 1 BitInformation measured as a choice between two options
- 2 CapacityA noisy channel has a top rate for reliable messages, 1948
- 3 CodesHamming builds the first practical error-correcting codes, 1950
- 4 Closing inTurbo codes come close to the limit, 1993
- 5 TodayPhones, fibre, Wi-Fi, streaming and the web