# Product Analytics: From Data to Decisions

> Turn product data into decisions you can defend. Define metrics that don't lie, read funnels, retention and cohorts, design and read A/B tests, work out LTV and payback, and forecast growth, all on realistic data from a bill-splitting app.

- Source: https://productovi.com/courses/product-analytics
- Free, no sign-up needed to read.

## Modules

### Module 1. What exactly are we measuring?

Events, properties and users; totals vs unique users and other counting traps; metric definitions two people can agree on; a tracking plan for a new feature.

- [Events, properties and users](https://productovi.com/courses/product-analytics/events-properties-users.md): The language of product data: what an event records and how event and user properties differ.
- [Totals, unique users and counting traps](https://productovi.com/courses/product-analytics/counting-traps.md): Why the same week of data gives different numbers of 'active users', and how to count them right.
- [Metric definition: one rule, one number](https://productovi.com/courses/product-analytics/metric-definitions.md): How to write a metric definition so two people get the same number: qualifying event, unit, exclusions, time window and time zone, on a DAU example.
- [A tracking plan for a new feature](https://productovi.com/courses/product-analytics/tracking-plan.md): Before payment reminders ship, plan the data: the questions, the events and properties that answer them, where each is logged, the metric definitions and the checks before launch.

### Module 2. Which numbers should we steer by?

North star and the metric tree; volume metrics, quality metrics and ratios that mislead; DAU, WAU, MAU and stickiness at the product's natural rhythm.

- [The north star metric and its KPI tree](https://productovi.com/courses/product-analytics/north-star.md): One north star metric for the value users get, and a KPI tree of input metrics that teams can actually move.
- [Volume, quality and Simpson's paradox](https://productovi.com/courses/product-analytics/volume-vs-quality-metrics.md): Tell how much from how well, and catch Simpson's paradox: a per-group ratio that rose only because the mix of groups changed.
- [DAU, WAU, MAU and stickiness](https://productovi.com/courses/product-analytics/active-users.md): Count active users at the rhythm people actually use the product.
- [Case: a north star for Halves](https://productovi.com/courses/product-analytics/choosing-a-north-star.md): Pick and defend a north star metric for Halves, with six months of numbers behind it.

### Module 3. Where do new users get stuck?

Funnels: order, conversion window and who counts; reading a funnel without fooling yourself; finding the activation moment.

- [Funnels: conversion window, step order and who counts](https://productovi.com/courses/product-analytics/funnels.md): How a funnel conversion rate is computed, and why the conversion window, the step order and who enters give different numbers on the same users.
- [Reading a funnel: right censoring and date ranges](https://productovi.com/courses/product-analytics/reading-funnels.md): Right censoring (users who haven't had time to convert yet), date ranges and other ways a funnel conversion rate misleads.
- [Activation: finding the aha moment](https://productovi.com/courses/product-analytics/activation.md): Find your product's aha moment: the week-one action that separates users who stay from users who leave, and know what that comparison can't prove.
- [Case: fix the onboarding funnel](https://productovi.com/courses/product-analytics/onboarding-funnel.md): Find where new Halves users drop off and what to fix first.

### Module 4. Do people come back?

Day-N, rolling and bracket retention; reading and averaging a cohort table; the retention plateau as a product-market-fit signal.

- [Retention: day N, rolling or bracket](https://productovi.com/courses/product-analytics/retention-types.md): Three ways to measure retention and which fits a product people use weekly or monthly.
- [Reading a cohort table](https://productovi.com/courses/product-analytics/cohort-table.md): Read rows, columns and diagonals, and average cohorts without the classic mistakes.
- [The plateau: retention and product-market fit](https://productovi.com/courses/product-analytics/retention-plateau.md): What a flattening retention curve says about the product, and what it doesn't.
- [Case: retention analysis by segment](https://productovi.com/courses/product-analytics/retention-by-segment.md): Is Halves retention healthy for each kind of group? Run a retention analysis by segment on a year of cohort data and write the verdict.

### Module 5. Did the change work?

Hypothesis, primary metric and a decision rule up front; test size and duration; who is in the test; reading a finished test honestly.

- [A/B test plan: hypothesis, metrics, guardrails](https://productovi.com/courses/product-analytics/test-design.md): Write the A/B test plan before launch: a hypothesis, one primary metric, guardrail metrics and a decision rule.
- [How big and how long](https://productovi.com/courses/product-analytics/test-size.md): Sample size and duration for a product test.
- [Who is in the test: triggers and cluster randomization](https://productovi.com/courses/product-analytics/who-is-in-the-test.md): Who belongs in an A/B test: triggered enrollment, dilution, and cluster randomization of whole groups when users affect each other.
- [Case: reading a finished test](https://productovi.com/courses/product-analytics/reading-a-test.md): Effect, interval, guardrails and the decision.

### Module 6. What can we learn without a test?

Fair version comparisons; traffic mix and seasonality; correlation, selection and 'users who do X retain better'.

- [Pre/post analysis: comparing versions fairly](https://productovi.com/courses/product-analytics/version-comparison.md): Pre/post analysis: the rules that make a before-and-after comparison of app versions worth reading.
- [Traffic mix and seasonality](https://productovi.com/courses/product-analytics/traffic-and-seasons.md): What seasonality and a changing traffic mix do to product metrics, and how to compare like with like.
- [Selection bias: users who do X retain better](https://productovi.com/courses/product-analytics/correlation-and-selection.md): A selection bias example: correlation, self-selection, a confounding variable and what a feature's users can and can't tell you.
- [Case: the version or the traffic?](https://productovi.com/courses/product-analytics/did-the-version-help.md): Did the new Halves version help, or did the users change?

### Module 7. Are we making money on it?

ARPU, conversion to paid and trial-to-paid; the LTV curve by cohort; CAC, payback and ROI by cohort and channel; pricing and paywall decisions.

- [How to calculate ARPU and ARPPU](https://productovi.com/courses/product-analytics/revenue-metrics.md): How to calculate ARPU and ARPPU, conversion to paid and trial-to-paid, and how a change in the paywall trigger mix fakes a better paywall.
- [The LTV curve](https://productovi.com/courses/product-analytics/ltv-curve.md): Cumulative gross profit per cohort user by month, how to read it, and when a young cohort can be extrapolated.
- [CAC, payback and ROI](https://productovi.com/courses/product-analytics/cac-payback-roi.md): Judge a paid channel by the cohort it buys: CAC, payback period and cohort ROI at a stated age, and why a month's profit divided by that month's spend flatters ads.
- [Price testing case: price or paywall?](https://productovi.com/courses/product-analytics/price-or-paywall.md): Price testing in practice: read a price test and a paywall before-and-after honestly, then recommend what Halves Plus should do next.

### Module 8. How will we grow?

Forecasting active users from cohorts; measuring virality honestly; channels, capacity and modern attribution; investigating an unexplained spike.

- [Forecasting active users](https://productovi.com/courses/product-analytics/forecasting.md): New users × retention: build a monthly active users forecast from cohorts, with a separate curve for summer cohorts, and price a cut in paid acquisition in users.
- [Invites and viral growth](https://productovi.com/courses/product-analytics/virality.md): Viral growth from invites: the viral coefficient k, cycle time, why k below 1 still pays, and how to measure invites without flattering yourself.
- [Channels and marketing attribution](https://productovi.com/courses/product-analytics/channels.md): Channel capacity and marginal CAC, a break-even cost that values the invitees a paid user brings, marketing attribution through the stores' privacy frameworks, and incrementality from a geo holdout.
- [Capstone: the unexplained spike](https://productovi.com/courses/product-analytics/spike-investigation.md): Sign-ups more than doubled over a weekend. List the possible causes, rule them out one by one with the data at hand, and write the memo.