Measures of spread: range, SD and quartiles

The standard deviation (SD) measures how far a typical value sits from the mean, in the units of the data. It takes six steps:

1. Find the mean.
2. Subtract it from each value to get the deviations.
3. Square each deviation, so negatives stop cancelling positives.
4. Add up the squares.
5. Divide by n − 1 to get the sample variance.
6. Take the square root.

Courier A's 10 deliveries average 24 hours. The deviations are −3, −3, −1, −1, 0, 0, +2, +2, +2, +2; the squares add up to 36; 36 / 9 = 4; √4 = 2 hours. Courier B, with the same mean, gets 576 / 9 = 64 and an SD of 8 hours.

Why n − 1? Ten deliveries are a sample, and the sample mean sits a little closer to its own values than the true long-run mean does. Dividing by 9 instead of 10 corrects for that. In a spreadsheet, STDEV.S is the sample version and STDEV.P the population one.

The median splits sorted data in half; quartiles split it into quarters. Q1 has a quarter of the values below it, Q2 is the median, Q3 has three quarters below it.

A simple way to find them by hand:

1. Sort the values.
2. Split them into a lower and an upper half.
3. Q1 is the median of the lower half, Q3 the median of the upper half.

Courier A, sorted: 21, 21, 23, 23, 24 | 24, 26, 26, 26, 26. The lower half's median is 23, the upper half's is 26.

The interquartile range (IQR) is Q3 − Q1 = 26 − 23 = 3 hours: the width of the middle 50% of deliveries. Courier B's IQR is 30 − 18 = 12 hours.

The IQR ignores the extremes completely, which makes it the natural spread partner of the median. Spreadsheet QUARTILE functions interpolate and can give slightly different quartiles on other data; that's fine, as long as you use one method when comparing groups.

To measure spread from every value, you start with deviations: each value minus the mean. Their plain average is useless, because the mean is the balance point of the data and the deviations always add up to zero.

The fix is to square them. Squares are never negative, and big misses count far more than small ones: a deviation of 14 becomes 196. The average of the squared deviations (dividing by n − 1 for a sample) is the variance. For courier B's 10 deliveries (mean 24 hours) it's 576 / 9 = 64.

Squaring has a side effect: the units are squared too. Courier B's variance is 64 hours², a unit nobody can picture. Order values would give a variance in "dollars squared".

That's why reports use the standard deviation, the square root of the variance: √64 = 8 hours, back in normal units. The two carry the same information. Variance is the convenient form inside formulas and tests; the SD is the one you put in front of people, written next to the mean: "24 ± 8 hours".

Each step of the standard deviation fixes a problem left by the step before it.

1. Mean: you need a centre to measure distances from.
2. Deviations: value minus mean, one distance per value. Their sum is always zero, so you can't stop here.
3. Squares: they remove the signs and make large misses count more.
4. Sum: one number for the whole dataset.
5. Divide by n − 1: turns the sum into an average squared distance, the sample variance. The n − 1 corrects for the sample mean sitting a bit too close to its own data. With the whole population, divide by n.
6. Square root: undoes the squaring of the units, giving the SD in hours, dollars or minutes.

Swapping steps breaks the result. Taking the root before dividing, or averaging the deviations before squaring them, gives a different number that has no meaning. A quick sanity check: the SD can't be negative, and for most data it is smaller than the range divided by 2.

Measures of spread differ in how much one extreme value can move them.

• Range uses only the two most extreme values, so one strange value changes it directly.
• Variance and SD use every value, and squaring makes big deviations count even more. One far-off value can dominate the whole sum of squares.
• IQR depends only on the middle half of the sorted data, so the extremes don't enter it at all.

Add one lost parcel of 90 hours to courier A's 10 deliveries. The range jumps from 5 to 69 hours. The SD jumps from 2 to about 20 hours, and the variance from 4 to about 400. The IQR stays at 3 hours, because Q1 and Q3 barely move.

That's why spread measures come in pairs with centre measures: mean with SD, median with IQR. When the data has a long tail or a few freak values, report the median and IQR (or percentiles) for the typical experience, and look at the extremes separately.

Is an SD of 8 big? It depends on the scale. Eight hours is a lot for a 24-hour delivery and nothing for a 30-day subscription cycle. And an SD in dollars can't be compared with an SD in sign-ups at all.

The coefficient of variation (CV) fixes both problems: divide the SD by the mean, usually as a percentage.

CV = SD / mean

Daily ad spend with a mean of $2,000 and an SD of $200 has a CV of 10%. Daily sign-ups with a mean of 50 and an SD of 15 have a CV of 30%. Spend is steady, but results swing three times as much relative to their size. That's a hint that something other than budget, such as the day of the week or the creatives, drives sign-ups.

Use CV only for values measured from a true zero: hours, money, counts. For metrics that can be negative or have an arbitrary zero, such as NPS or temperature, the ratio is meaningless.

"24 ± 8 hours" is the most common way to report a centre with its spread. It's worth reading it carefully.

The SD is the typical distance from the mean, not a minimum or a maximum. Some values are closer, some are further away. For data that forms a roughly symmetric hill, a rule of thumb helps: about 68% of values fall within one SD of the mean and about 95% within two.

Courier B: one SD around 24 hours is 16 to 32 hours, and 7 of the 10 deliveries fall there. Two SDs is 8 to 40 hours, and all ten do. The 36-hour promise sits inside the two-SD band, so occasional late deliveries are expected, not a fluke.

Two cautions. First, the rule assumes a bell-ish shape; for skewed data, use percentiles instead. Second, ten values are too few to trust the percentages exactly. Still, "24 ± 8 hours" tells a manager far more than "24 hours on average": it says how much a customer's experience can differ from the headline number.

There are two versions of the variance and the SD, and they differ only in step 5.

• Sample (the default in product work): divide the sum of squares by n − 1. Use it when your data is a slice of something bigger: five weeks of a programme that will run for months, 500 orders out of all orders.
• Population: divide by n. Use it only when you have every value you care about and don't want to generalise.

The sample version is a little larger. That's on purpose: the sample mean is computed from the same values, so the deviations from it come out slightly too small, and n − 1 compensates.

With 5 weekly values of 12, 15, 9, 18 and 6, the mean is 12 and the squares add up to 90. Sample: 90 / 4 = 22.5, SD ≈ 4.74. Population: 90 / 5 = 18, SD ≈ 4.24.

The gap shrinks as n grows, but with small samples it is noticeable. When a dashboard shows an SD, check which version it uses.

A percentile is the value below which a given share of the data falls. The 90th percentile, or p90, is the time within which 90% of deliveries arrive; the remaining 10% take longer. Quartiles are special percentiles: Q1 is p25, the median is p50, Q3 is p75.

Product teams use high percentiles for promises and service levels: "p90 delivery time is 30 hours", "p95 page load is 2.8 seconds". They answer the question averages can't: how bad is it for the customers who wait longest?

Compare the alternatives:

• the mean describes the centre, not the tail;
• the maximum is one value, so a single lost parcel becomes the whole metric;
• the SD mixes fast and slow deviations and swings with one extreme value.

A high percentile sits between them: it focuses on the slow end, yet one freak value among hundreds hardly moves it. With very few values (say 10 deliveries) p90 is shaky, so collect enough data before setting a target on it.