# 95% confidence intervals

> Building a 95% confidence interval from its formula and margin of error, so you report a range of plausible values instead of one number.

- Course: [Statistics for Product Managers and Marketers](https://productovi.com/courses/statistics-for-pm-and-marketers.md)
- Module: A/B testing in practice
- Lesson 9/24, ~40 min
- Source: https://productovi.com/courses/statistics-for-pm-and-marketers/confidence-intervals

## One number isn't the whole story

**Alex Tutor:** New week at CatChow, and you're on the product team's numbers again. What have you got?

**Sofia:** Last week's average order value was \$50.00. So our AOV is \$50, right? Case closed?

**Alex Tutor:** Almost. \$50.00 is the average of *that sample*. A different week's sample would give a slightly different number, even if nothing really changed.

![Three small bowls scooped from the same big sack of cat food, each filled to a slightly different level, while a cat compares them.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/confidence-intervals-illustration-2.webp)

*Same shop, same week, different samples: each one gives a slightly different average.*

**Sofia:** So the real AOV could be anything?

**Alex Tutor:** Not anything — but a range of plausible values around \$50.00. That range is a **confidence interval (CI)**: instead of one number, we report an interval likely to contain the true average. Today we'll build the usual one, a 95% confidence interval.

> **Confidence interval (CI)**
>
> A range of values, computed from a sample, that is likely to contain the true population parameter (here, the true average order value).

![A cat casts a wide fishing net over a pond, and a single orange fish swims inside the area the net covers.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/confidence-intervals-illustration-3.webp)

*A confidence interval is a net, not a spear: it covers a range that very likely holds the true value.*

> **Exercise:** A 95% confidence interval means there is a 95% probability that the true average order value lies inside this specific interval.

## Confidence interval formula for a mean

**Alex Tutor:** Building one is mechanical. First: the **standard error (SE)** — how much the sample mean would bounce around from sample to sample.

$$
SE = \frac{s}{\sqrt{n}}
$$

*Standard error of the mean: sample standard deviation s divided by the square root of the sample size n.*

**Sofia:** So a bigger sample gives a smaller SE?

**Alex Tutor:** Exactly — SE shrinks with $\sqrt{n}$, not n itself. Quadrupling your sample only halves the SE.

**Alex Tutor:** Take last week's 225 orders: sample SD \$15.00, so SE = \$15.00 / √225 = \$15.00 / 15 = \$1.00.

> **Exercise:** This month CatChow sampled n = 400 orders with sample standard deviation s = \$20.00. What is the standard error of the mean, in dollars?

**Alex Tutor:** Now the full interval: point estimate ± (critical value × SE). For a 95% CI, the critical value is about 1.96 (from the normal distribution).

$$
\bar{x} \pm z_{\alpha/2} \cdot SE
$$

*95% CI for a mean, using the normal critical value z = 1.96.*

**Alex Tutor:** Back to last week: \$50.00 ± 1.96 × \$1.00 = \$50.00 ± \$1.96, so the 95% confidence interval runs from \$48.04 to \$51.96.

**Sofia:** So the margin of error there is just the \$1.96 part?

**Alex Tutor:** Right — the **margin of error** is critical value × SE. Report it alongside the point estimate; it's the honest picture leadership actually needs.

> **Exercise:** Using a critical value of 1.96 and SE = \$2.50, what is the margin of error, in dollars?

**Alex Tutor:** Now put all three pieces together on a fresh sample: SE, margin, then the interval itself.

> **Exercise:** A sample of n = 144 CatChow orders has a mean of \$52.40 and a standard deviation of s = \$24. What is the **lower** bound of the 95% confidence interval for the average order value, in dollars?

## 95% vs 99% confidence: the precision trade-off

**Sofia:** Can we just always use 99% confidence to be extra sure?

**Alex Tutor:** You can, but the interval gets wider. Higher confidence means a bigger critical value, which means a bigger margin of error.

**Margin of error grows with confidence level (SE = \$1.00)**

| Confidence level | Margin of error (USD) |
| --- | --- |
| 90% | 1.645 |
| 95% | 1.96 |
| 99% | 2.576 |

**Alex Tutor:** At 99% confidence that same \$1.00 SE gives a ±\$2.58 margin instead of ±\$1.96 — a wider, less precise interval.

**Sofia:** So if I want both high confidence AND a narrow interval...

**Alex Tutor:** ...you need a bigger sample. SE shrinks with $\sqrt{n}$, so a narrower interval at the same confidence level means collecting more data, not wishing harder.

> **Exercise:** To make a 95% CI narrower without lowering the confidence level, what should you do?

**Alex Tutor:** Let's make sure that sticks — which levers actually narrow a CI, and which just look like they should?

**Sofia:** Some things obviously narrow it — bigger n. But higher variability confuses me.

**Alex Tutor:** More variability in the data (bigger s) makes SE bigger, which widens the CI — the opposite of what you want.

**Sofia:** And a non-random, cherry-picked sample?

**Alex Tutor:** That doesn't just fail to narrow the CI — it makes the whole interval meaningless, because the formula assumes a random sample.

![A cat picks only the orange pieces out of a bowl of mixed blue and orange kibble and drops them into a separate small dish.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/confidence-intervals-illustration-4.webp)

> **Exercise:** Which of these narrow a 95% CI, holding everything else fixed? Select all that apply.

## Confidence interval for a proportion: a preview

**Alex Tutor:** Everything so far used a *mean* (dollars). CatChow also tracks *proportions* — like conversion rate. The idea is the same, the SE formula changes slightly.

$$
SE_p = \sqrt{\frac{p(1-p)}{n}}
$$

*Standard error of a sample proportion p, from a sample of size n.*

**Alex Tutor:** Same shape as before: point estimate ± critical value × SE. We'll build a full example — including the significance test — in the next two lessons.

**Sofia:** Okay, I think I've got the mechanics. Can we practice the order of steps?

**Alex Tutor:** Sure — walking through the steps out of order is a good way to see if it's actually stuck, not just memorised.

> **Exercise:** Put the steps of building a confidence interval in the right order.

**Alex Tutor:** And one quick try at the proportion version, since conversion is what the next two lessons are about.

> **Exercise:** Last week 2,400 visitors reached CatChow’s subscription page and 4% of them subscribed. What is the standard error of that proportion, in percentage points?

## How to interpret and report a confidence interval

**Alex Tutor:** One more thing before you go present this. Leadership sometimes only wants the single number — the CI feels like unnecessary hedging to them.

**Sofia:** Isn't a single number just... simpler?

**Alex Tutor:** Simpler, yes. Honest, no. \$50.00 with no context invites overconfidence: someone might compare it to \$49.50 next month and call that a 'drop.'

**Sofia:** Even though \$49.50 is well inside this month's interval.

**Alex Tutor:** Exactly. So the wording on your slide matters. Which of these two sentences would you put there?

> **Exercise:** Last week’s AOV is \$50.00 with a 95% CI of \$48.04 to \$51.96 (orders vary with s = \$15). Which sentence would you put on the slide?

**Alex Tutor:** Good. Now your turn to put it in your own words for someone who's never heard of a CI.

> **Exercise:** Sofia, a product manager at CatChow, needs to explain to a marketing manager, in 2-3 sentences, why CatChow should report the confidence interval for AOV (e.g. \$48.04-\$51.96), not just the \$50.00 average. Write her explanation.

**Alex Tutor:** Nice. One more - a slightly trickier situation to reason through.

**Sofia:** Bring it on.

**Alex Tutor:** CatChow's finance lead saw this month's AOV interval, \$48.04-\$51.96, and last month's, \$47.50-\$50.10, and asked if AOV is now definitely higher.

**Sofia:** Overlapping intervals... that's the trick, isn't it?

> **Exercise:** Has CatChow's AOV really increased?

**Alex Tutor:** Good instinct - flagging the overlap without over-claiming either way. That's the core skill this module builds on.

**Sofia:** Next up: how big a sample do we actually need before we even start?

**Alex Tutor:** Exactly the right question. That's power and sample size - next lesson.

- Previous lesson: [p-value and statistical significance](https://productovi.com/courses/statistics-for-pm-and-marketers/p-value.md)
- Next lesson: [Power analysis and sample size](https://productovi.com/courses/statistics-for-pm-and-marketers/power-and-sample-size.md)

---
Productovi Learning. Free, no sign-up needed to read.