# Information theory

Information can be measured in bits, and every channel has a maximum rate for error-free messages.

Source page: https://www.materialinsights.org/science/shannon_information/

Mathematics and information · Information theory · No. 126

- Electronic age
- Science
 Mathematics and information · The science behind technology

Claude Shannon showed in 1948 that every noisy channel has a top speed for error-free messages, and that clever coding can come close to it.

Chart: Illustration: Information theory

##### Where it comes from

Chart: Map of where it comes from

Shannon, a mathematician at Bell Labs, published his theory in two parts in July and October 1948. It began with the problem of reproducing at one point a message chosen at another.

##### Three things to know

1. The 1948 paper appears to contain the first published use of the word bit.
2. Shannon's rule says a message can be sent with as few errors as you like at any rate below capacity, but not above it.
3. Gallager invented low-density parity-check codes by 1962, and they were largely forgotten for more than 30 years.

##### The journey

1. 1 **Bit**Information measured as a choice between two options
2. 2 **Capacity**A noisy channel has a top rate for reliable messages, 1948
3. 3 **Codes**Hamming builds the first practical error-correcting codes, 1950
4. 4 **Closing in**Turbo codes come close to the limit, 1993
5. 5 **Today**Phones, fibre, Wi-Fi, streaming and the web
 Covered in [Heat, temperature and the laws of thermodynamicsScience](https://www.materialinsights.org/learn/science/heat-and-temperature/)

##### Go deeper pick a tab to open it in full

## Other sections

- [The idea](https://www.materialinsights.org/science/shannon_information/sci-idea/index.md)
- [Discovery](https://www.materialinsights.org/science/shannon_information/sci-discovery/index.md)
- [Technologies](https://www.materialinsights.org/science/shannon_information/sci-technologies/index.md)
