# Paired t-test: before and after

> Comparing the same users before and after a change with a paired t-test or the Wilcoxon signed-rank test.

- Course: [Statistics for Product Managers and Marketers](https://productovi.com/courses/statistics-for-pm-and-marketers.md)
- Module: Many groups and before/after
- Lesson 15/24, ~40 min
- Source: https://productovi.com/courses/statistics-for-pm-and-marketers/paired-tests

## Paired vs unpaired t-test: same customers, twice

**Alex Tutor:** CatChow launched a loyalty program last month, and Marketing has brought you, on the product team, a question: did it actually raise monthly spend, or did spend just drift up anyway?

**Sofia:** Easy: compare average spend last month vs this month with a t-test, like we did for our A/B groups.

**Alex Tutor:** Careful: that independent-samples t-test compares two averages built from whichever customers happened to order in each month, not the same people.

**Alex Tutor:** Here we tracked the exact same 8 customers, before and after. That's a paired (related) samples design, and it deserves its own test: the paired t-test.

![The same cat shown twice, linked by an arc: first beside a half-empty shopping basket, then beside the same basket, now fuller.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/paired-tests-illustration-2.webp)

*Paired data: the same customer before and after, so each one is their own baseline.*

> **Paired (related) samples**
>
> Measurements that come in matched pairs, typically the same units measured twice, such as customers before and after a change.

**Sofia:** Why does it matter that it's the same people?

**Alex Tutor:** Because each customer becomes their own baseline. A paired test only looks at each person's own change, removing the noise from customers who simply spend differently from one another.

> **Exercise:** Why is a paired t-test more appropriate than an independent-samples t-test for comparing the same 8 customers' spend before and after the loyalty program?

## The customers, before and after

**Alex Tutor:** Monthly spend before → after, USD, for our 8 loyalty customers: U1 42→47, U2 38→40, U3 51→59, U4 29→30, U5 60→66, U6 33→37, U7 47→50, U8 55→62.

**Change in monthly spend per customer**

| Customer | After − before |
| --- | --- |
| U1 | 5 |
| U2 | 2 |
| U3 | 8 |
| U4 | 1 |
| U5 | 6 |
| U6 | 4 |
| U7 | 3 |
| U8 | 7 |

*Every customer's difference (after minus before) is positive, though the size of the change varies quite a bit.*

**Alex Tutor:** Look at the differences: every single customer spent more after the program, some just a little, some a lot.

**Sofia:** So it obviously worked?

**Alex Tutor:** Looks promising, but "obviously" isn't a number. We still need to test whether the average difference is bigger than plain noise.

> **Exercise:** A paired t-test can compare "before" measurements from one group of customers with "after" measurements from a completely different group of customers.

## Paired t-test formula: one number per customer

**Alex Tutor:** An independent-samples t-test splits its degrees of freedom across two groups: n1 + n2 − 2. Paired data collapses to one group of differences instead.

**Alex Tutor:** Each customer becomes one number: their own after-minus-before difference. Then it's a simple question: is the mean difference zero?

$$
t = \dfrac{\bar{d}}{s_d / \sqrt{n}}
$$

*The paired t-test: mean difference divided by the standard error of the difference (SD of differences over the square root of n pairs).*

**Alex Tutor:** Our 8 differences, in dollars: 5, 2, 8, 1, 6, 4, 3, 7.

**Alex Tutor:** Their sum is 36, so the mean difference is 36 / 8 = 4.5 dollars.

> **Exercise:** The loyalty program's before/after differences (after − before) for the 8 customers are: 5, 2, 8, 1, 6, 4, 3, 7 dollars. Compute the mean difference.

**Alex Tutor:** Now the spread: squared deviations from 4.5 sum to 42. Dividing by n − 1 = 7 gives a variance of 6, so the standard deviation of the differences is about 2.45.

**Alex Tutor:** The standard error divides that by the square root of n: 2.45 / √8 ≈ 0.87 dollars.

**Sofia:** So how big does the t-statistic need to be to matter?

**Alex Tutor:** For df = 7, a two-tailed 5% test has a critical value of about 2.365. Anything beyond that, in absolute value, is surprising if the program truly changed nothing.

> **Exercise:** For the 8 loyalty-program differences (5, 2, 8, 1, 6, 4, 3, 7), the sample standard deviation of the differences is about 2.449. Compute the standard error of the mean difference.

## Is 4.5 dollars real or noise?

**Alex Tutor:** Your turn first: put the mean difference over its standard error.

> **Exercise:** Mean difference \$4.50, standard error of the mean difference \$0.866, 8 customers. Compute the paired t-statistic (two decimals).

**Alex Tutor:** t = 4.5 / 0.87 ≈ 5.20, with df = 7. That's more than double the critical value of 2.365.

**Alex Tutor:** We reject H0: the mean before/after difference is very unlikely to be zero. This 4.5 dollar rise looks real, not noise.

**Alex Tutor:** Meanwhile a colleague ran a different test on the same two columns and got a very different answer. Which one do you trust?

> **Exercise:** CatChow tracked the same 8 customers' monthly spend before and after its loyalty program. A colleague ran an independent-samples t-test on the before and after columns and got t ≈ 0.76, p ≈ 0.46. The paired test on the same data gives t ≈ 5.20, p ≈ 0.001. Which conclusion is right?

**Sofia:** So the loyalty program caused it?

**Alex Tutor:** One caveat: with no control group, a seasonal trend could also explain part of it. The paired test proves the change is real, not that this one cause fully explains it.

![A cat carries a full shopping basket past a window with falling snow and a string of festive orange lights.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/paired-tests-illustration-3.webp)

*No control group means the calendar could be doing part of the work.*

> **Exercise:** Match each term to what it describes.

## Not normal? The Wilcoxon signed-rank test

**Alex Tutor:** The paired t-test assumes the differences are roughly normal. With badly skewed differences or a tiny sample, that assumption gets shaky.

**Alex Tutor:** The sign test is the simplest fallback: it only counts how many differences are positive versus negative, ignoring their size entirely.

**Alex Tutor:** Try it on our data: every one of the 8 differences is a plus. How surprising is that if nothing changed?

> **Exercise:** CatChow tracked 8 customers' monthly spend before and after its loyalty program, and all 8 differences (after − before) are positive. You run a sign test. If the program changed nothing, each difference would be equally likely to be positive or negative. What is the probability that all 8 come out positive, as a percentage (two decimals)?

![A row of small cats: most raise a paw next to an orange up arrow, one next to a blue down arrow; all the arrows are the same size.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/paired-tests-illustration-4.webp)

**Alex Tutor:** The Wilcoxon signed-rank test keeps a bit more information: it ranks the absolute differences and compares the rank sums of the positive and negative shifts.

**Sofia:** So Wilcoxon is usually the better pick over the sign test?

> **Exercise:** Which statements about paired designs and paired tests are true? Select all that apply.

## Wrap-up

**Alex Tutor:** Usually yes: Wilcoxon uses more of the data, so it has more power, while the sign test is the simplest, most robust fallback when even ranks feel shaky.

**Alex Tutor:** Whichever test you pick, always check the raw differences and the sample size first. Eight customers is enough for a clean lesson, not always for a confident launch decision.

**Sofia:** Noted: same people, one column of differences, then pick the test that matches how those differences look.

**Alex Tutor:** Two tasks left: put pairing's advantage into your own words, then a trickier case with a tiny, lopsided sample.

> **Exercise:** In 2–4 sentences, explain why pairing before/after measurements on the same 8 customers gives more statistical power than comparing two independent groups of 8 different customers each.

**Alex Tutor:** Now for the case: a redesigned help centre, 6 users, satisfaction rated 1 to 10 before and after.

**Alex Tutor:** No clean t = 5.20 this time. One score jumps suspiciously far, and the sample is tiny.

**Sofia:** Bring it on.

**Alex Tutor:** That's the spirit.

> **Exercise:** What test would you use, and how would you handle the outlier?

- Previous lesson: [Two-way ANOVA and interactions](https://productovi.com/courses/statistics-for-pm-and-marketers/multifactor-anova.md)
- Next lesson: [Repeated measures ANOVA](https://productovi.com/courses/statistics-for-pm-and-marketers/repeated-measures.md)

---
Productovi Learning. Free, no sign-up needed to read.