# Sample vs population and the normal distribution

> Sample vs population: why we study samples to learn about all users, why the bell-shaped normal distribution shows up everywhere, and how the central limit theorem ties them together.

- Course: [Statistics for Product Managers and Marketers](https://productovi.com/courses/statistics-for-pm-and-marketers.md)
- Module: Comparing two groups
- Lesson 5/24, ~39 min
- Source: https://productovi.com/courses/statistics-for-pm-and-marketers/samples-and-normal-distribution

## Sample vs population: what's the difference?

**Alex Tutor:** Morning, and welcome back to CatChow's product team. Before we compare any two groups, we need to agree what we're even measuring: the difference between a sample and a population. Grab coffee, this part is short but it trips everyone up.

**Sofia:** Sure. We're measuring CatChow orders, right? I pulled last month's 500 orders into a sheet already.

**Alex Tutor:** Those 500 are a **sample**: the part we actually looked at. The **population** is every order that exists or could exist — every customer who ever bought, or ever will.

**Sofia:** So the population is basically infinite and we can never fully measure it?

**Alex Tutor:** Pretty much — future customers haven't even shown up yet. So we use a sample to make educated guesses about the population. The trick is making sure the sample actually represents it.

![A cat chef tastes a single spoonful from a huge pot of stew to judge the whole pot.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/samples-and-normal-distribution-illustration-2.webp)

*One spoonful can tell you about the whole pot, if the pot is well stirred.*

**Sofia:** What could go wrong? I just took the most recent 500 orders.

**Alex Tutor:** That's actually fine if orders arrive in no particular pattern. But if you'd only sampled orders placed through our app, you'd miss every desktop shopper. That's **sampling bias**: a systematic reason some units are more likely to be picked.

> **Exercise:** CatChow's data team wants to survey a random 10% of everyone who ordered in the last year and use the result to describe satisfaction across *all* customers who order this year. The average score computed from the replies is a...

**Alex Tutor:** Good. Now, two more words you'll hear constantly: a number that describes the whole population (like the true average spend of every CatChow customer) is a **parameter**. It's usually unknown — nobody has measured every customer.

**Sofia:** And the number we compute from our 500 orders is...?

**Alex Tutor:** A **statistic**. Same idea, different source: computed from a sample, used to *estimate* the parameter. Statisticians love giving two names to one concept depending on where it came from.

**Sofia:** So our sample mean of \$45 is a statistic, and the 'true' average order value of every CatChow customer ever is a parameter we'll never know exactly.

**Alex Tutor:** Exactly. Match these four terms to their definitions before we move on — it's the vocabulary the rest of this module leans on.

> **Exercise:** Match each term to its definition.

## What is a normal distribution?

**Alex Tutor:** Now, why does everyone keep drawing that bell-shaped curve? Plot the order values of thousands of independent customers and, remarkably often, you get a symmetric hump: the **normal distribution**.

**Sofia:** Why would order values just happen to form a bell shape?

**Alex Tutor:** Loosely: each order value is nudged by many small, independent factors — mood, stock, coupons, cat appetite. Add up enough small random nudges around a typical value and a bell curve tends to emerge.

![Kibble pellets fall from a funnel through rows of pegs, bounce left and right and pile up into a symmetric bell-shaped heap while a cat watches.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/samples-and-normal-distribution-illustration-3.webp)

*Many small random pushes add up to a bell shape.*

**Alex Tutor:** In a perfect normal curve, mean, median and mode all sit at the same point, right in the middle, and the curve is symmetric on both sides.

**The standard normal curve**

| z-score | Density |
| --- | --- |
| -3 | 0.004 |
| -2.5 | 0.018 |
| -2 | 0.054 |
| -1.5 | 0.13 |
| -1 | 0.242 |
| -0.5 | 0.352 |
| 0 | 0.399 |
| 0.5 | 0.352 |
| 1 | 0.242 |
| 1.5 | 0.13 |
| 2 | 0.054 |
| 2.5 | 0.018 |
| 3 | 0.004 |

*A normal distribution with mean 0 and standard deviation 1. About 95% of the area sits between z = -2 and z = 2.*

**Sofia:** Is that the 68-95-99.7 thing I've seen in a footnote somewhere?

> **The empirical rule**
>
> In a normal distribution, about 68% of values fall within 1 standard deviation of the mean, about 95% within 2, and about 99.7% within 3.

**Alex Tutor:** True or false time, then we'll put a number on 'how many standard deviations away'.

> **Exercise:** The empirical rule says that in a normal distribution, about 95% of values fall within 2 standard deviations of the mean.

**Sofia:** Which sampling method would actually give us a biased read on customer satisfaction, by the way? I want to make sure I don't repeat my app-only mistake.

**Alex Tutor:** Fair question — bias sneaks in whenever the *chance of being picked* correlates with the thing you're measuring. Sometimes it's obvious, sometimes it hides in something as innocent as timing.

**Sofia:** Give me an example of the hidden kind?

**Alex Tutor:** Surveying only people who just contacted support: you'll oversample the frustrated and undersample the happily silent majority. Good instinct to double-check. Pick every biased option below — there might be more than one.

> **Exercise:** Which of these sampling methods would give CatChow a biased read on customer satisfaction? Select all that apply.

## z-scores

**Alex Tutor:** Back to the bell curve. A **z-score** tells you how many standard deviations a single value sits from the mean. It turns any normal variable into the same universal scale.

$$
z = \frac{x - \mu}{\sigma}
$$

*z-score: subtract the population mean μ from the value x, then divide by the population standard deviation σ.*

**Sofia:** So a z-score of 0 means 'exactly average', and a z-score of 2 means 'two standard deviations above average'?

**Alex Tutor:** Exactly. Negative z-scores sit below the mean. Combine a z-score with the empirical rule and you instantly know roughly how unusual a value is.

**Alex Tutor:** Say CatChow's full population of orders has mean $\mu = \$50$ and standard deviation $\sigma = \$12$. Where does an order of \$74 sit?

> **Exercise:** CatChow's full population of orders has mean $\mu = \$50$ and standard deviation $\sigma = \$12$. What is the z-score of an order worth \$74?

## Sampling distributions and the central limit theorem

**Alex Tutor:** Here's the part that makes hypothesis tests work at all. Imagine drawing thousands of *different* samples of 40 orders each and computing the mean of every one.

**Sofia:** Those means would all be a bit different from each other, I guess?

**Alex Tutor:** Right — that spread of sample means is called the **sampling distribution** of the mean. And here's the beautiful part: whatever shape the original orders have, that distribution of means tends toward normal as the sample size grows.

**Sofia:** Even if individual order values are skewed, with a long tail of big bulk orders?

**Alex Tutor:** Even then. That's the **central limit theorem** (CLT): sample means become approximately normal as $n$ grows, regardless of the population's own shape. It's *the* reason means are so convenient to work with.

![A cat takes many scoops from a lopsided heap of kibble and drops one marble per scoop into a tray, where the marbles form a symmetric bell shape.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/samples-and-normal-distribution-illustration-4.webp)

*Averages of many samples line up into a bell, even when the raw data is lopsided.*

> **Exercise:** The central limit theorem says the sampling distribution of the mean becomes approximately normal as sample size grows, even if the underlying population is skewed.

## Standard error

**Alex Tutor:** One last piece. The spread of that sampling distribution has its own name: the **standard error (SE)**. Don't confuse it with the standard deviation of the raw data.

$$
SE_{\bar{x}} = \frac{s}{\sqrt{n}}
$$

*Standard error of the mean: the sample standard deviation s, divided by the square root of the sample size n.*

**Sofia:** So SE shrinks when the sample gets bigger? That feels like 'more data, less uncertainty about the mean'.

**Alex Tutor:** Precisely — and because of the square root, you need 4x the data to halve the standard error, not 2x. Diminishing returns on sample size, always worth remembering before you ask for 'just a bigger sample'.

**Alex Tutor:** A few more numbers, then we wrap up.

> **Exercise:** A sample of n = 36 orders has sample standard deviation s = \$18. What is the standard error of the mean?

**Alex Tutor:** Now turn it around. Say \$3 isn't precise enough and you want the standard error cut in half. Remember the square root.

> **Exercise:** Your sample of n = 36 orders has s = \$18, so the standard error is \$3. Assuming s stays about the same, how many orders would you need to bring the standard error down to \$1.50?

**Alex Tutor:** Last one before the wrap-up, and it's the slip I see most in reports: reading the standard error as if it described individual orders.

> **Exercise:** A sample of 36 orders has a mean of \$45, s = \$18 and a standard error of \$3. Which sentence reads the standard error **correctly**?

## Wrap-up

**Alex Tutor:** Let's recap: population and sample, parameter and statistic, the normal curve, z-scores, and now standard error. That's the full toolkit for the rest of this module.

**Sofia:** It's a lot of vocabulary for one morning. Can we test it on something real before I forget it all?

**Alex Tutor:** Happy to. Our head of growth walked past my desk this morning and said something that made me wince.

**Alex Tutor:** He said: 'Sample mean is $45, so the true average for every customer is $45.' In your own words, what's wrong with that sentence?

> **Exercise:** Our head of growth said: 'The sample mean is $45, so the true average order value for every customer is $45.' In 2-3 sentences, what's wrong with this, and what should we say instead?

**Alex Tutor:** Nicely put. You've got the vocabulary now — let's see you apply it to something messier than a one-line quote.

**Sofia:** Messier how? Please tell me it's not another one of my own decisions coming back to haunt me.

**Alex Tutor:** Funny you should mention that. Remember the checkout redesign you shipped last week?

**Sofia:** Oh no.

> **Exercise:** Review Sofia's launch-day claim.

- Previous lesson: [Types of charts: choosing the right one](https://productovi.com/courses/statistics-for-pm-and-marketers/charts.md)
- Next lesson: [Two-sample t-test](https://productovi.com/courses/statistics-for-pm-and-marketers/t-test.md)

---
Productovi Learning. Free, no sign-up needed to read.