standard error (SE)

The standard deviation of a statistic's sampling distribution; for a mean it is SD / sqrt(n).

The standard error (SE) measures how much a statistic, such as a sample mean or a conversion rate, would wobble if you repeated the same study on a fresh sample. It is the standard deviation of that statistic's sampling distribution. For a mean:

$SE = \frac{s}{\sqrt{n}}$

where $s$ is the sample standard deviation and $n$ the sample size. For a proportion $p$ (say, a conversion rate), $SE = \sqrt{p(1-p)/n}$ .

The key is not to confuse it with the standard deviation. They answer different questions:

	Standard deviation (SD)	Standard error (SE)
Describes	individual values (orders, customers)	an estimate (a mean, a rate, a lift)
Question	how different are customers from each other?	how precisely do we know the average?
More data	settles near its true value, doesn't shrink	shrinks, in proportion to $1/\sqrt{n}$

The SD is a property of your customers; the SE is a property of your measurement. A bigger sample doesn't make customers more alike, but it does pin down the average more tightly.

The SE is the building block of inference. A 95% confidence interval is roughly estimate $\pm 1.96 \times SE$ , and t- and z-statistics divide an observed difference by its SE. Because of the square root, halving the SE takes four times the data, not twice as much.

Example

CatChow's analyst looks at 225 orders from last week: mean $50.00, standard deviation $15.00.

$SE = \frac{15}{\sqrt{225}} = \frac{15}{15} = \$1.00$

The two numbers say very different things:

SD = $15: individual orders differ a lot. Plenty of orders are around $35 or $65; that's just how customers buy.
SE = $1: the average is known quite precisely. The 95% confidence interval is $50 \pm 1.96 \times 1 = \$48.04$ to $\$51.96$ .

Now the team collects four times as many orders, $n = 900$ , and the SD comes out at about $15 again. The SE becomes $15 / \sqrt{900} = 15 / 30 = \$0.50$ and the interval narrows to $50 \pm 0.98$ , i.e. $49.02 to $50.98. Customers are just as varied as before; only our knowledge of the mean improved, and halving the SE took four times the data.

The same logic for a rate: 60 checkouts out of 1,000 sessions is a 6.0% conversion rate with $SE = \sqrt{0.06 \times 0.94 / 1{,}000} \approx 0.0075$ , or 0.75 percentage points.

Common mistakes

Reporting the SE as if it described customers. "Order value varies by ±$1" is wrong when the SD is $15; the $1 is the uncertainty of the average, not the spread of orders.
Not saying which one is on the chart. Error bars can show the SD, the SE or a 95% CI; they differ several times in length. Always label them.
Expecting the SD to shrink with more data. Only the SE shrinks. If the SD keeps falling as the sample grows, check how the data are being collected.
Doubling the sample to halve the uncertainty. Because of $\sqrt{n}$ , you need four times the sample for half the SE.
Trusting a small SE from a biased sample. The SE only covers random sampling noise; it says nothing about a sample drawn only from app users or only from the launch day.

Learn it in the course

Sample vs population and the normal distribution · Sample vs population: why we study samples to learn about all users, why the bell-shaped normal distribution shows up everywhere, and how the central limit theorem ties them together.
Two-sample t-test · The two-sample t-test (Student's t-test): comparing the means of two groups and deciding whether the gap is bigger than chance.
95% confidence intervals · Building a 95% confidence interval from its formula and margin of error, so you report a range of plausible values instead of one number.
Paired t-test: before and after · Comparing the same users before and after a change with a paired t-test or the Wilcoxon signed-rank test.
Conversion rates and the two-proportion z-test · Comparing conversion rates between variants with a two-proportion z-test and reading uplift correctly.