Articles about Multiple testing

Articles for product managers and marketers about Multiple testing: plain explanations, examples and common mistakes.

October 2, 202610 min read

Peeking in A/B testing: why stopping early fails and how sequential testing fixes it

Peeking in A/B testing turned a 5% false-positive rate into 27.6% in 100,000 simulated tests. When to stop an A/B test, and how sequential testing helps.

A/B test design
Statistical significance

October 2, 202610 min read

Bonferroni correction in A/B/n testing: compare several variants without a false winner

The Bonferroni correction on a four-variant A/B/n test: why multiple comparisons create false winners, how Holm compares, and what it costs in traffic.

Multiple testing
Post-hoc comparisons