# How science knows: observation, models, measurement and testing: Uncertainty: how sure can a measurement be

Source page: https://www.materialinsights.org/learn/science/how-science-knows/p5/

[All courses](https://www.materialinsights.org/learn/) / [Understanding Science](https://www.materialinsights.org/learn/science/) / [Module 1: How science knows: observation, models, measurement and testing](https://www.materialinsights.org/learn/science/how-science-knows/) Part 5 of 6

## Uncertainty: how sure can a measurement be

No measurement is exact. A good scientist reports a number together with how far off it could be, and designs the work so that the doubt shrinks. This part separates precision from accuracy, shows how to write and shrink uncertainty, and explains how scientists tell a real effect from luck.

### 5.1 Every measurement has an error, and the two kinds behave differently

Weigh the same packet of tea five times on a kitchen scale and you may read 252, 249, 251, 250 and 253 grams. The packet did not change. The readings vary because the scale, your reading and the surface are never quite steady. This scatter is called random error. It pushes readings above and below the true value by chance, and it can be reduced by repeating and averaging.

Now suppose the scale was set up with a small offset, so that it reads 6 grams too high on every packet. All your readings are off by the same amount in the same direction. This is systematic error, a bias built into the instrument or the method. Averaging does not remove it. A thousand readings will cluster beautifully around the wrong value.

This difference explains the two words that people often swap. Precision describes how tightly repeated readings cluster, which is the size of the random error. Accuracy describes how close the average is to the true value, which depends on the systematic error. A scale can be precise but inaccurate, as with the offset above, or accurate on average but imprecise, as with a shaky one. The best instrument is both, and the two are checked in different ways. Precision is judged by repeating. Accuracy is judged by comparing with a trusted standard, such as a certified 500 g weight.

Systematic errors are the more dangerous, because nothing in the data warns you. In 2011 a badly connected fibre-optic cable in the OPERA neutrino experiment made particles seem faster than light. The error was found and the result withdrawn. A clock that runs fast, a thermometer in direct sun, a questionnaire with leading wording and a survey that only reaches people with phones are all of this type.
 Worked example

#### Why a scale needs a zero check

A jeweller's scale should read 0.00 g with nothing on it. If it reads 0.03 g, every weighing is 0.03 g too heavy. On a 2 g gold chain that is 1.5%, about Rs 300 of gold at an illustrative Rs 10,000 per gram. Shopkeepers and inspectors in India check scales against certified weights for exactly this reason: the error is silent and always pushes the same way.

- Hewitt, Lyons, Suchocki and Yeh, Conceptual Integrated Science, 2nd edition (Pearson, 2013) · ch. 1, 'About Science'
- [Scientific American, Superluminal neutrino result caused by faulty connection? (2012)](https://www.scientificamerican.com/blog/life-unbounded/superluminal-neutrino-result-caused-by-faulty-connection/) · web

### 5.2 Reporting a result honestly means giving a range and not too many digits

A measurement is incomplete without its uncertainty. 'The table is 1.50 m long' leaves open whether you can trust the second decimal place. 'The table is 1.50 plus or minus 0.01 m' says that the true length is probably between 1.49 and 1.51. The plus-or-minus figure is not an apology. It is information, and without it one cannot say whether two measurements agree.

Take the packet of tea. The five readings average 251 g, and the individual readings spread by about 2 g. The average of five is better than any single reading. A standard result of statistics says that the uncertainty in the mean shrinks roughly with the square root of the number of readings. With 5 readings the improvement is about 2.2 times, with 100 readings it is 10 times. Taking 100 times the readings gives only 10 times the precision, which is why the search for very high precision is so costly. And it works only for random error. If the scale is biased by 6 grams, 10,000 readings will not help.

Significant figures are a short-hand for the same honesty. If a measurement is good to three digits, write three. Writing 251.0000 g because the calculator displayed it would claim a precision you do not have. When you multiply or divide, the answer cannot be more reliable than the least reliable input. The engineering module Thinking in estimates makes the same point: keep only the digits the data can justify.

In science, whether two numbers disagree depends on their uncertainties. A measured 9.78 plus or minus 0.02 m/s^2 for gravity in your town and a standard 9.81 differ by 0.03, about one and a half times the uncertainty, which is unremarkable. If the uncertainty were 0.002, the same gap would signal that something real, such as latitude or altitude, was going on.

Data Worked estimate: five readings of a packet of tea (grams)

| Step | Result | Working |
|---|---|---|
| Readings | 252, 249, 251, 250, 253 | Five weighings |
| Mean | 251 g | 1255 / 5 (calculated) |
| Spread of a single reading (standard deviation) | about 1.6 g | Square root of (1+4+0+1+4) / 4 (calculated) |
| Uncertainty of the mean | about 0.7 g | 1.6 / square root of 5 = 1.6 / 2.2 (calculated) |
| Report | 251.0 plus or minus 0.7 g | One decimal is enough |
| Readings for plus or minus 0.1 g | about 250 | (1.6 / 0.1)^2 = 256 (calculated) |

Averaging cures random scatter only. A 6 g offset would remain.
 Source: Hewitt, Lyons, Suchocki and Yeh, Conceptual Integrated Science, 2nd edition (Pearson, 2013), ch. 1, 'About Science'

- Hewitt, Lyons, Suchocki and Yeh, Conceptual Integrated Science, 2nd edition (Pearson, 2013) · ch. 1, 'About Science'

### 5.3 Statistics separates a real effect from luck, and correlation from cause

A coin lands heads six times in a row. Is it biased? Not necessarily: the chance of six heads is 1 in 64, and in a long evening of flipping you would see it. The same question arises whenever data are noisy. A new medicine seems to help 60 of 100 patients and a dummy helps 52. Is the medicine working, or is that gap the sort of luck you would see anyway? Statistics supplies a way to put a number on it, by asking how often chance alone would produce a gap that large.

A common measure is a count of 'sigma', the number of standard widths of random scatter by which the result sits from what chance expects. Particle physicists demand five sigma, which corresponds to a chance of about 1 in 3.5 million of a false alarm, before they announce a discovery. The LIGO detection of gravitational waves on 14 September 2015, at more than 5.1 sigma, and the Higgs particle in 2012 both met this bar. In medicine and social science the bar is often lower, so more published claims fail to hold up when repeated.

Opinion polls give a feel for how size matters. A properly random poll of n people has an uncertainty of roughly one divided by the square root of n. For 1,000 people that is 1 / 32, about 3 percentage points. To halve the margin you need four times the people. Polls in India's huge, varied electorate show why the sample must be random and not simply large: a million replies from an online form would be biased towards people with internet access.

Last, correlation is not cause. Ice cream sales and drowning rise together, because both rise in summer. A third factor, heat, drives both. Data can show that two things move together. To argue that one causes the other, you need an experiment, a natural experiment or a mechanism, and you need to rule out a hidden third factor.
 Common misreading

#### 'Five sigma means the theory is 99.99997% likely to be true'

Five sigma is about how rarely random scatter alone would make a result this extreme if there were no effect. It is not the probability that the hypothesis is correct. A five-sigma result can still come from a hidden systematic error, which is why different teams, using different instruments, check the claim.

- Hewitt, Lyons, Suchocki and Yeh, Conceptual Integrated Science, 2nd edition (Pearson, 2013) · ch. 1, 'About Science'
- [Abbott et al. (LIGO Scientific Collaboration and Virgo), Observation of Gravitational Waves from a Binary Black Hole Merger, Physical Review Letters 116, 061102 (2016)](https://doi.org/10.1103/PhysRevLett.116.061102) · web
- [Abbott et al., GW150914: First results from the search for binary black hole coalescence with Advanced LIGO, Physical Review D 93, 122003 (2016)](https://ui.adsabs.harvard.edu/abs/2016PhRvD..93l2003A) · web

### See it in the table
[Au Gold](https://www.materialinsights.org/commodities/gold/)[Tea Tea](https://www.materialinsights.org/commodities/tea/)

### Check your understanding

A bathroom scale reads 2 kg too high every time, with readings very close together. This scale is
 The readings cluster tightly, so it is precise. The constant offset means they miss the true value, so it is not accurate. Averaging cannot remove this kind of bias.

You average 100 readings in place of 4. Roughly how much smaller is the random uncertainty of the mean?
 Random uncertainty falls as the square root of the number of readings. The ratio of the roots, 10 to 2, gives a factor of 5.

A random poll of 2,500 people has a margin of about
 One over the square root of 2,500 is 1/50, which is 0.02, or 2 percentage points.

[← Part 4: Measurement, units and powers of ten](https://www.materialinsights.org/learn/science/how-science-knows/p4/)[Mark read · Part 6: Limits of science, and science with technology and society →](https://www.materialinsights.org/learn/science/how-science-knows/p6/)

## Other sections

- [Module overview](https://www.materialinsights.org/learn/science/how-science-knows/index.md)
- [What science is and where it came from](https://www.materialinsights.org/learn/science/how-science-knows/p1/index.md)
- [Observation, hypothesis and experiment](https://www.materialinsights.org/learn/science/how-science-knows/p2/index.md)
- [Facts, models, laws and theories](https://www.materialinsights.org/learn/science/how-science-knows/p3/index.md)
- [Measurement, units and powers of ten](https://www.materialinsights.org/learn/science/how-science-knows/p4/index.md)
- [Limits of science, and science with technology and society](https://www.materialinsights.org/learn/science/how-science-knows/p6/index.md)
- [Key terms, quiz and sources](https://www.materialinsights.org/learn/science/how-science-knows/review/index.md)
