95% confidence intervals

A single sample average is a point estimate: one number that would come out a bit different from a different week's orders. A confidence interval (CI) adds the honest part: a span of plausible values for the true population average.

The tricky bit is the 95%. It describes the procedure, not one particular interval. Imagine repeating the whole thing many times: draw a fresh random sample, build an interval the same way, again and again. About 95% of those intervals would contain the true average, and about 5% would miss it.

Once you've computed your interval, say $48.04 to $51.96, the true average is fixed. It's either inside or outside, and you don't know which. So the correct wording is "we're 95% confident the interval contains the true average", meaning the method works 95% of the time.

In day-to-day reporting this nuance rarely changes a decision. It matters when someone treats the interval as a guarantee, or reads 95% as the share of customers inside it.

Every confidence interval for a mean starts with the standard error (SE): how much the sample mean would move from one random sample to the next.

SE = s / √n

Here s is the sample standard deviation (how much individual orders differ) and n is the number of orders. With s = $20 and n = 400: SE = 20 / √400 = 20 / 20 = $1.00. Individual orders swing by about $20, but the average of 400 of them is pinned down to about a dollar.

Two things to notice:

• the square root. Precision grows slowly: four times the orders only halves the SE;
• the standard deviation doesn't shrink with more data. It describes your customers; the SE describes your uncertainty about their average.

The most common slip is dividing by n instead of √n. With the numbers above that gives 20 / 400 = $0.05, an SE twenty times too small, and every interval built on it will look far more precise than the data allow.

The standard error tells you how much the mean wobbles. To turn it into an interval you need to decide how sure you want to be, and that's what the critical value does.

For a 95% interval based on the normal distribution, the critical value is about 1.96: roughly 95% of sample means land within 1.96 standard errors of the true mean. Multiply the two and you get the margin of error:

margin of error = critical value × SE

With SE = $2.50: 1.96 × 2.50 = $4.90. With SE = $1.00 it's simply $1.96.

The margin of error is the "±" part of the interval, and it's the number worth saying out loud in a meeting: "AOV is $50, give or take about $5". It's not the same as the SE. Quoting the SE as if it were the margin makes the result look about twice as precise as it is at 95% confidence. So put the margin of error on the slide and keep the SE for your working.

With the standard error and the critical value in hand, the interval is one line:

x̄ ± critical value × s / √n

Take a sample of 144 orders with x̄ = $52.40 and s = $24:

• SE = 24 / √144 = 24 / 12 = $2.00;
• margin of error = 1.96 × 2.00 = $3.92;
• interval = 52.40 ± 3.92, from $48.48 to $56.32.

The interval is symmetric: the point estimate sits exactly in the middle, and the margin is the distance to each end. That gives you a quick self-check: the two bounds should average back to x̄ ((48.48 + 56.32) / 2 = 52.40).

Watch which number you subtract. Taking away the SE (52.40 − 2.00 = 50.40) gives the edge of a roughly 68% interval, not a 95% one, and makes the interval look half as wide as it should. Taking away s (52.40 − 24) mixes up the spread of single orders with the uncertainty of the average. For 99%, swap 1.96 for 2.576 and the bounds move further apart.

Once the confidence level is fixed, the critical value is fixed too (1.96 for 95%). The only part left that you can change is the standard error, SE = s / √n.

You can't really choose s: it reflects how different your customers' orders are. That leaves n. A bigger random sample shrinks the SE, and with it the margin of error.

Because of the square root, the price climbs fast. With s = $15:

• n = 225 gives SE = 15 / 15 = $1.00 and a margin of ±$1.96;
• n = 900 gives SE = 15 / 30 = $0.50 and a margin of ±$0.98.

Four times the orders bought half the width. That's worth knowing before you promise a stakeholder a "more precise number by Friday".

Cosmetic changes don't help. Rounding $50.00 to $50 doesn't make it more certain, and reporting the standard deviation instead of the interval answers a different question (how varied orders are) rather than how well you know the average.

The width of a confidence interval comes from one expression: critical value × s / √n. Each piece pushes in a predictable direction:

• sample size n: bigger n, smaller SE, narrower interval;
• confidence level: 90% uses a critical value of about 1.645, 95% uses 1.96 and 99% about 2.576. Lower confidence gives a narrower interval, at the price of missing the true value more often;
• variability s: more spread in the data, bigger SE, wider interval. You usually can't lower s by choice, though cleaner metrics (fewer bot orders, a fixed time window) sometimes help;
• how the sample was drawn: not in the formula at all, but the formula assumes a random sample. A hand-picked or convenience sample can produce a neat, narrow interval around the wrong value.

So when an interval looks surprisingly tight, check the last point first. Precision from a biased sample is worse than an honest wide interval, because it invites confident wrong decisions.

Every confidence interval in this course follows the same six steps, whether it's for an average order value or a conversion rate:

1. Collect a random sample. Everything after this assumes it.
2. Compute the point estimate: the sample mean or the sample proportion.
3. Compute the standard error: s / √n for a mean, √(p(1 − p) / n) for a proportion.
4. Choose the confidence level and look up its critical value: 1.96 for 95%.
5. Multiply the critical value by the SE to get the margin of error.
6. Add and subtract the margin from the point estimate.

With last week's orders: x̄ = $50.00, s = $15, n = 225, so SE = $1.00, the margin is 1.96 × 1.00 = $1.96 and the interval runs from $48.04 to $51.96.

The order matters for a reason: if step 1 fails, steps 2 to 6 still produce tidy numbers, but they describe the sample you picked, not your customers.

Conversion rate, click-through rate and churn are proportions: the share of people who did something. The interval recipe stays the same; only the standard error changes:

SE = √(p(1 − p) / n)

where p is the sample proportion as a decimal and n the number of people. You don't need s here: for a yes/no outcome the spread follows from p itself.

With 2,400 visitors and a 4% conversion rate: 0.04 × 0.96 = 0.0384; divided by 2,400 that's 0.000016; the square root is 0.004, or 0.4 percentage points. The 95% interval is 4% ± 1.96 × 0.4 = 4% ± 0.78 pp, roughly 3.2% to 4.8%.

Two habits keep this clean. Keep p as a decimal inside the formula and convert to percentage points at the end. And remember the whole fraction p(1 − p) / n sits under the root: dividing by n outside it shrinks the SE by a factor of about 49 here. The next lessons use exactly this SE to compare two conversion rates.

A confidence interval earns its place in a report when it stops people from over-reading small changes. A good line has three parts: the point estimate, the range, and what the interval is about.

"Average order value: $50.00 (95% CI $48.04–$51.96)." Spoken: "about $50, and the true average is plausibly anywhere from $48 to $52."

Two misreadings come up again and again:

• the interval describes customers. It doesn't. It describes the average. Individual orders vary with the standard deviation, which here is $15, so single orders spread much wider;
• overlap settles comparisons. If next month's interval overlaps this one, that doesn't prove nothing changed, and if they barely don't overlap, that's not a formal test either. Comparing two periods needs a test of the difference, which comes later in this module.

The payoff is practical: when next month shows $49.50, the interval tells everyone that's well within normal noise, not a "drop".