A standardised effect size: difference in means divided by the pooled standard deviation.
Example
Common mistakes
Treating 0.2 / 0.5 / 0.8 as a law. They're rough conventions from behavioural research, not a measure of business value.
Dividing by the standard error instead of the standard deviation. SE shrinks with sample size, so you get an inflated "d" that is really just a t-statistic in disguise.
Comparing d across very different populations. A narrow segment (say, only subscription customers) has a smaller SD, which makes the same dollar gap look like a bigger d.
Ignoring skew and outliers. A few very large orders inflate the SD and shrink d; consider a log scale, a trimmed mean or a rank-based measure for heavy-tailed revenue data.
Using d for conversion rates. For yes/no outcomes, report the lift in percentage points (or Cohen's h), not d.
Cohen's d expresses the gap between two group means in units of standard deviation:
Here sp is the pooled standard deviation, a weighted blend of both groups' spread. When the groups are the same size it simplifies to sp=(s12+s22)/2. A d of 0.5 means the two means sit half a typical spread apart.
Because d has no units, you can compare effects on very different metrics (order value in dollars, session time in minutes, a 1–10 satisfaction score) and feed it into power calculations. For a two-sample t-test at α = 0.05 with 80% power you need roughly n≈16/d2 users per group, so halving d quadruples the sample.
Cohen offered rough labels: 0.2 small, 0.5 medium, 0.8 large. He meant them as a fallback when nothing better is known, not as a verdict on value. In product analytics a d of 0.05 on revenue per user can be worth millions, while a d of 1 on a vanity metric may be worth nothing, so always translate back into natural units too.
d is closely tied to Student's t-statistic. With equal groups of size n, t=d×n/2: t grows with the sample, d does not. That's exactly why t (and the p-value) answers "is it real?" while d answers "how big is it?". For conversion rates, report the lift in percentage points instead (or use Cohen's h, the proportion counterpart).
CatChow's upsell-widget test: 10 orders per group.
Control: mean $42.50, s1≈$3.03 (variance ≈9.17)
Widget: mean $52.50, s2≈$3.37 (variance ≈11.39)
Pooled SD (equal groups): sp=29.17+11.39=10.28≈$3.21.
Cohen's d: d=3.2152.50−42.50=3.2110.00≈3.12.
Cross-check with the t-test: t=3.12×10/2=3.12×2.236≈6.97, the same t as in the course's t-test lesson.
A d of 3.1 is enormous, far beyond the "large" mark of 0.8. It's large because order values in this small synthetic sample barely vary, so a $10 gap is more than three typical spreads.
Now imagine real CatChow orders, where the standard deviation of order value is closer to $25. The same $10 gap gives d=10/25=0.4, between small and medium. Planning a test to detect it: n≈16/0.42=16/0.16=100 orders per group.
