# Mann–Whitney: when data isn't normal

> A rank-based test for skewed metrics like revenue and session length.

- Course: [Statistics for Product Managers and Marketers](https://productovi.com/courses/statistics-for-pm-and-marketers.md)
- Module: Comparing two groups
- Lesson 7/24, ~41 min
- Source: https://productovi.com/courses/statistics-for-pm-and-marketers/mann-whitney

## A test with an outlier problem

**Alex Tutor:** New scenario, same CatChow product team: we redesigned the product page and measured how long visitors spend on it: 6 sessions on the old page, 6 on the new one.

**Sofia:** Great, t-test time! I already have the means.

**Alex Tutor:** Not so fast. Look at the raw numbers first: old page is 38, 42, 47, 49, 55, 61 seconds. New page is 58, 60, 65, 68, 72... and 130.

**Sofia:** 130 seconds? That one visitor must have gone to make coffee and left the tab open.

![A laptop is left open on a cat-food product page beside a steaming mug while a cat walks away; an hourglass on the desk keeps running.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/mann-whitney-illustration-2.webp)

**Alex Tutor:** Exactly the kind of thing that wrecks a mean — remember outliers from the first module? That single session would drag the new page's average way up, making the 't' from a t-test unreliable here.

**Sofia:** So what do we do, just delete that session and pretend it didn't happen?

**Alex Tutor:** Tempting, but there's a cleaner fix: a test that only cares about *order*, not raw magnitude, so one extreme value can't dominate it. Time to meet the **Mann-Whitney U test**.

**Alex Tutor:** Quick check before we dive in.

> **Exercise:** Which situation is the clearest reason to prefer Mann-Whitney over a classic t-test?

## Ranking everyone together

**Alex Tutor:** Step one: throw all 12 sessions into one pile, sort them smallest to largest, and give each one a **rank** — 1 for the shortest, 12 for the longest.

**All 12 sessions, sorted and ranked**

| Rank | Old page | New page |
| --- | --- | --- |
| 1 | 38 |  |
| 2 | 42 |  |
| 3 | 47 |  |
| 4 | 49 |  |
| 5 | 55 |  |
| 6 |  | 58 |
| 7 |  | 60 |
| 8 | 61 |  |
| 9 |  | 65 |
| 10 |  | 68 |
| 11 |  | 72 |
| 12 |  | 130 |

*Bar height is session duration; position is its rank (1 = shortest). Old-page sessions cluster at low ranks, new-page sessions at high ranks — except one outlier that doesn't change the overall pattern.*

**Sofia:** The new page mostly sits at the high ranks — except one old-page session at rank 8, sneaking in above two new-page ones.

**Alex Tutor:** Good eye. Real data is rarely perfectly clean. Step two: add up the ranks that belong to each group. That total is called the **rank sum**.

**Alex Tutor:** Old page got ranks 1, 2, 3, 4, 5, 8 — a rank sum of 23. New page got the other six ranks, summing to 55. Check: 23 + 55 = 78, and 1 through 12 also sums to 78. Good, nothing lost.

**Alex Tutor:** Time to turn those rank sums into the actual test statistic.

> **Exercise:** In CatChow's product page test, all 12 sessions (6 on the old page, 6 on the new) are ranked 1 (shortest) to 12 (longest). The old page's ranks are 1, 2, 3, 4, 5 and 8. What is its rank sum R₁?

## From rank sums to U

$$
U_1 = R_1 - \frac{n_1(n_1+1)}{2}, \quad U_2 = R_2 - \frac{n_2(n_2+1)}{2}, \quad U = \min(U_1, U_2)
$$

*Each group's U subtracts out the rank sum it would have if it only ranked against itself. The smaller of the two is the Mann-Whitney U statistic.*

**Sofia:** That n(n+1)/2 term looks familiar from the rank-sum check we just did.

**Alex Tutor:** Good catch, it's the same formula — the sum of ranks 1 through n. Subtracting it from a group's actual rank sum tells you how much that group 'over-performed' compared to ranking purely against itself.

**Sofia:** So a bigger U for a group means its values tended to rank higher overall?

**Alex Tutor:** Right — U₁ and U₂ each describe one group's own showing. It's only once we take the *smaller* of the two that we get the actual test statistic. You already have R₁ = 23 for the old page, with n₁ = 6. Compute U₁.

> **Exercise:** Old page: R₁ = 23, n₁ = 6. What is U₁ = R₁ - n₁(n₁+1)/2?

**Sofia:** U₁ = 2 — tiny. Does that already tell us the old page barely 'wins' any pairwise comparison against the new one?

**Alex Tutor:** Exactly that intuition — U literally counts, roughly, how many times a value from one group beats a value from the other. Now the mirror image: same recipe for the new page.

![A grid of one-on-one matchups: in each pair a blue cat and an orange cat stand on a small scale, and the orange cat wins almost every time.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/mann-whitney-illustration-3.webp)

*U counts pairwise wins between the two groups.*

**Sofia:** R₂ = 55, n₂ = 6?

**Alex Tutor:** Exactly. Go for it.

> **Exercise:** New page: R₂ = 55, n₂ = 6. What is U₂ = R₂ - n₂(n₂+1)/2?

**Alex Tutor:** So U = min(2, 34) = 2. Here's the twist: for Mann-Whitney, **small** U is what signals a difference — the smaller it is, the less the two groups' ranks overlap.

**Sofia:** That's the opposite of the t-test, where a *bigger* number was the exciting one.

> **The decision rule runs backwards**
>
> Unlike the t-test, you reject H0 when the calculated U is *less than or equal to* the critical value from the table, not greater than. For n₁ = n₂ = 6, the two-tailed critical value at α = 0.05 is 5.

**Alex Tutor:** Worth repeating because it trips people up constantly: small U, not big U, is the exciting result here. Keep that backwards rule somewhere visible.

**Sofia:** Our U is 2, and the critical value is 5. 2 is less than 5, so... the page redesign really did change things?

> **Exercise:** With U = min(2, 34) = 2 and a two-tailed critical value of 5 (n₁ = n₂ = 6, α = 0.05), we reject the null hypothesis that the two pages have the same distribution of session length.

**Alex Tutor:** Significant, yes. Now the part people get wrong in the write-up: what exactly did we just show?

> **Exercise:** CatChow's product page test (6 sessions on the old page, 6 on the new) came out significant with Mann-Whitney (U = 2 ≤ 5). Which sentence belongs in the report?

## When to reach for the rank-based test

**Alex Tutor:** Nicely done getting through the mechanics. Before we talk about when this test earns its keep versus a regular t-test, let's lock in the vocabulary.

**Sofia:** Fine by me, I'd rather not mix up 'rank' and 'rank sum' in a meeting.

**Alex Tutor:** Wise — half of statistical embarrassment is just mixing up two similarly-named things in front of an audience.

**Alex Tutor:** Match these ideas, then.

> **Exercise:** Match each term to its definition.

**Alex Tutor:** Mann-Whitney is a **non-parametric test**: it makes no assumption about the shape of the data, unlike the t-test, which is **parametric** and leans on approximate normality.

**Sofia:** So should we just always use Mann-Whitney and skip worrying about normality entirely?

**Alex Tutor:** Tempting, but no — when data genuinely is roughly normal with no nasty outliers, the t-test is more powerful: better at detecting a real effect from the same amount of data. Save the rank-based test for when skewness or outliers are actually a problem.

**Alex Tutor:** One more scenario where ranks are your *only* option, not just the safer one.

> **Exercise:** Which of these are good reasons to reach for Mann-Whitney instead of a t-test? Select all that apply.

**Alex Tutor:** One wrinkle you'll hit in real data before we wrap up: two values that are exactly equal. Here's how ranking handles a tie.

> **Exercise:** A tiny test of reviews read before buying. Old page: 10, 14 and 20 seconds. New page: 14, 25 and 30 seconds. Rank all six together, giving tied values the average of their ranks. What is U₁ for the old page?

## Wrap-up

**Sofia:** One thing still bugs me: why not just remove the 130-second session and run a normal t-test on what's left?

**Alex Tutor:** You could, and sometimes people do document that choice explicitly. But deleting inconvenient data is a slippery habit, and a real 130-second engaged session might be exactly the kind of user the redesign was meant to create.

**Sofia:** So ranking is really a way of saying 'I trust that this happened, I just don't trust the exact number' — is that fair?

**Alex Tutor:** That's a great way to put it. It's a deliberate trade: you give up some precision (exact seconds) to gain robustness (immunity to that one wild number).

![Cats stand on evenly spaced steps sorted by size; a giant cat on the top step is only one step above the next cat despite being much bigger.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/mann-whitney-illustration-4.webp)

*Ranking keeps who came first and drops how far ahead they were.*

**Alex Tutor:** In your own words: why does ranking the data, rather than deleting or averaging it, let us keep that outlier honestly in the analysis?

> **Exercise:** In the product page test, one new-page session lasted 130 seconds (most likely a tab left open), while the others ran about 40 to 70 seconds. In your own words, why does ranking the data (rather than deleting the 130-second session or leaving it to distort the mean) let us handle that outlier honestly?

**Alex Tutor:** Solid. Last one — a colleague's Mann-Whitney write-up needs a second pair of eyes before it reaches their manager.

**Sofia:** Let me guess, they mixed up the decision rule?

**Alex Tutor:** No spoilers. Read it yourself, the same way you'd want someone to check your work.

**Sofia:** Fair. Judging someone else's math is somehow more fun than judging my own, anyway.

> **Exercise:** Review a colleague's Mann-Whitney write-up.

- Previous lesson: [Two-sample t-test](https://productovi.com/courses/statistics-for-pm-and-marketers/t-test.md)
- Next lesson: [p-value and statistical significance](https://productovi.com/courses/statistics-for-pm-and-marketers/p-value.md)

---
Productovi Learning. Free, no sign-up needed to read.