# Other curves and power

> t, chi-square and F as the same idea with another table, degrees of freedom, expected counts and Σ(O − E)²/E, and computing the power of a test.

- Course: [Math for Statistics and Unit Economics](https://productovi.com/courses/math-basics.md)
- Module: Probability and distributions
- Lesson 30/36, ~36 min
- Source: https://productovi.com/courses/math-basics/other-curves-and-power

## Same idea, another table

**Alex Tutor:** Last lesson of the module, still on Klasna's product team. In the statistics course you’ll meet t-tests, chi-square tests and ANOVA with its F. They look like three new subjects. They’re one idea with three tables.

**Sofia:** Which idea?

**Alex Tutor:** Every test turns the data into one number, the **statistic**: roughly signal ÷ noise. Then it asks how far into the tail of its own curve that number lands. If beyond the point that leaves 5%, the **critical value**, the result counts as significant.

**Alex Tutor:** For z it’s 1.96. For t with 18 degrees of freedom, 2.101. For chi-square with 1 degree of freedom, 3.84. For F with (3, 20), 3.10. You don’t compute these; you read them from a table or a spreadsheet.

**Alex Tutor:** Try the idea on one t-test before the picture.

> **Exercise:** A t-test compares two groups of 10 learners (18 degrees of freedom) and gives t = 2.05. The 5% critical value for t with 18 df is 2.101. What is the verdict at 5%?

![Four curves with their 5% critical values: z 1.96, t with 18 df 2.101, chi-square with 1 df 3.84, F with (3, 20) df 3.10.](https://productovi.com/courses/math-basics/diagrams/m7-l6-four-curves.png)

*Beyond the dashed line is the 5% the test calls "significant".*

**Sofia:** And the degrees of freedom pick which curve?

**Alex Tutor:** Yes, roughly how much independent information is left. Two groups of 10 for a t-test: 10 + 10 − 2 = 18. A table with r rows and c columns: (r − 1)(c − 1). Four groups of 6 for F: (4 − 1, 24 − 4) = (3, 20).

**Alex Tutor:** Small samples give t heavier tails, so its critical value is above 1.96. As the degrees of freedom grow, t turns into z: at 100 it’s already 1.98.

> **Exercise:** Match each test to its statistic’s curve and degrees of freedom.

> **Exercise:** A t-test compares 12 learners from one group with 15 from another. How many degrees of freedom does it have?

**Alex Tutor:** And a quick check on what happens to t as samples grow.

> **Exercise:** Even with 100 degrees of freedom, t’s 5% critical value stays far above z’s 1.96, so large samples still need much stricter t tables.

## Chi-square: observed vs expected

**Alex Tutor:** Back to the goal-setting call: with it, 105 of 300 bought; without it, 125 of 500. Overall 230 of 800 bought. If the call made no difference, both groups would buy at 230 ÷ 800 = 28.75%.

**Alex Tutor:** That gives the **expected count** of each cell: row total × column total ÷ grand total. With a call and bought: 300 × 230 ÷ 800 = 86.25. We saw 105.

$$
\chi^2 = \sum \frac{(O - E)^2}{E}, \qquad E = \frac{\text{row total} \times \text{column total}}{\text{total}}
$$

*O: observed count; E: expected count if the groups didn’t differ. Add over all cells.*

**Alex Tutor:** Each cell adds (O − E)² ÷ E: 4.076 + 1.645 + 2.446 + 0.987 ≈ 9.15. That’s far beyond 3.84, so the difference is significant. And the z-test on the same data gives z ≈ 3.026, and 3.026² ≈ 9.15.

![Each cell compares observed with expected counts; the four (O − E)²/E add up to about 9.15, above the 3.84 cut-off and equal to z².](https://productovi.com/courses/math-basics/diagrams/m7-l6-chi-square.png)

*For a 2×2 table, chi-square is z squared, and 3.84 is 1.96 squared.*

**Sofia:** So 3.84 is just 1.96 squared?

**Alex Tutor:** For one degree of freedom, yes: 1.96² = 3.84. Two tests, one answer. Chi-square earns its keep on bigger tables, say three packages × two age groups, where a single z doesn’t fit.

> **Exercise:** Of 200 Club members who joined a speaking club in week one, 30 cancelled; of 600 who didn’t, 150 cancelled. If joining made no difference, what is the expected number of cancellations among the 200 who joined?

**Alex Tutor:** Now the whole statistic, and a shortcut for 2×2 tables.

> **Exercise:** A two-proportion z-test on a 2×2 table of Klasna learners gives z = 2.5. What chi-square value would the chi-square test give on the same table?

## Computing power

**Alex Tutor:** Last tool: **power**, the chance that a test catches an effect that is really there. With a two-tailed 5% test and a true effect δ measured with standard error SE, power ≈ P(Z > 1.96 − δ ÷ SE).

$$
\text{power} \approx P\left(Z > 1.96 - \frac{\delta}{SE}\right)
$$

*δ: the true effect; SE: the standard error of the measured difference.*

**Sofia:** Where does that come from?

**Alex Tutor:** If the effect is real, the measured difference sits δ ÷ SE standard errors from zero on average. It’s significant when it lands past 1.96. So power is the area of the shifted curve beyond 1.96. When δ ÷ SE = 2.8, you get P(Z > −0.84) = 80%.

**Alex Tutor:** The product-analytics course has a paywall test with 1,700 users per variant, 4% vs 5%. SE = √(0.04 × 0.96 ÷ 1,700 + 0.05 × 0.95 ÷ 1,700) ≈ 0.0071. δ ÷ SE = 0.01 ÷ 0.0071 ≈ 1.41. Power = P(Z > 0.55) ≈ **29%**.

> **Exercise:** A real effect of 2 pp is measured with SE = 0.8 pp in a two-tailed 5% test. Use P(Z > −0.54) = 0.705, P(Z > 0.54) = 0.295 and P(Z > 1.42) = 0.078. What is the power, in percent?

**Alex Tutor:** Last check: how would you raise that 29%?

> **Exercise:** The paywall test had 1,700 users per variant, SE ≈ 0.0071 for a true lift from 4% to 5%, and power of about 29%. Which change raises the power to about 80% for the same effect?

**Sofia:** So with 29% power, "not significant" mostly means "we didn’t look hard enough".

> **Exercise:** The paywall test ran a week: 1,700 users per variant, power about 29% for +1 pp. The result is "not significant", and a teammate says: "So the new paywall doesn’t work." In two or three sentences, explain why that conclusion is too strong.

## Case: read a reminder test end to end

**Alex Tutor:** To close the module, one test from start to finish: counts, expected counts, chi-square, its table, and the power the test had.

> **Exercise:** Write a short readout of the reminder test for Klasna’s growth team.

## Where you will meet this

> **In short**
>
> - Every test: a **statistic** compared with a **critical value** from its own curve. z 1.96; t (df 18) 2.101; χ² (df 1) 3.84; F (3, 20) 3.10.
> - **Degrees of freedom:** n₁ + n₂ − 2 for t; (r − 1)(c − 1) for a table; (k − 1, N − k) for F.
> - **Chi-square** = Σ(O − E)²/E with E = row × column ÷ total. For a 2×2 table, χ² = z².
> - **Power** ≈ P(Z > 1.96 − δ/SE). 1,700 per variant for 4% → 5%: 29%. δ/SE = 2.8 gives 80%.

> **These lessons use it**
>
> - [Two-sample t-test](/courses/statistics-for-pm-and-marketers/t-test): df = 18 and the critical value 2.101.
> - [Mann–Whitney: when data isn't normal](/courses/statistics-for-pm-and-marketers/mann-whitney): another statistic with its own table.
> - [Conversion rates and the two-proportion z-test](/courses/statistics-for-pm-and-marketers/conversion-rate-tests): z for two shares.
> - [One-way ANOVA: comparing many groups](/courses/statistics-for-pm-and-marketers/anova): F with (3, 20) and 3.10.
> - [Course recap: choosing the right test](/courses/statistics-for-pm-and-marketers/choosing-the-right-test): chi-square for tables of counts.
> - [How big and how long](/courses/product-analytics/test-size): 29% power with 1,700 users per variant.

**Alex Tutor:** That closes probability. Next comes the bridge to unit economics.

- Previous lesson: [The normal curve](https://productovi.com/courses/math-basics/the-normal-curve.md)
- Next lesson: [Contribution margin and break-even](https://productovi.com/courses/math-basics/contribution-and-break-even.md)

---
Productovi Learning. Free, no sign-up needed to read.