Skip to content
Log in
  • Trainer
  • Product managers
  • Marketers

Product Analytics: From Data to Decisions

Turn product data into decisions you can defend. Define metrics that don't lie, read funnels, retention and cohorts, design and read A/B tests, work out LTV and payback, and forecast growth, all on realistic data from a bill-splitting app.

Free32 lessons~17 h 36 min of learning

A big phone shows a receipt split in two halves next to friends sharing a pizza, with a funnel, a line chart and a cell grid floating above.
Start the course

  1. Module 1: What exactly are we measuring?

    Events, properties and users; totals vs unique users and other counting traps; metric definitions two people can agree on; a tracking plan for a new feature.

    A magnifying glass hovers over a phone with a split receipt as a stream of small colored dots flows out of the screen.
    1. 1.1Events, properties and usersThe language of product data: what an event records and how event and user properties differ.~28 min

      1.1 Events, properties and users

      ~28 min · 9 exercises

      StartRead theory
    2. 1.2Totals, unique users and counting trapsWhy the same week of data gives different numbers of 'active users', and how to count them right.~32 min

      1.2 Totals, unique users and counting traps

      ~32 min · 10 exercises

      StartRead theory
    3. 1.3Metric definition: one rule, one numberHow to write a metric definition so two people get the same number: qualifying event, unit, exclusions, time window and time zone, on a DAU example.~31 min

      1.3 Metric definition: one rule, one number

      ~31 min · 9 exercises

      StartRead theory
    4. 1.4A tracking plan for a new featureBefore payment reminders ship, plan the data: the questions, the events and properties that answer them, where each is logged, the metric definitions and the checks before launch.~37 min

      1.4 A tracking plan for a new feature

      ~37 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  2. Module 2: Which numbers should we steer by?

    North star and the metric tree; volume metrics, quality metrics and ratios that mislead; DAU, WAU, MAU and stickiness at the product's natural rhythm.

    A bright orange star shines above a phone with a split receipt, with dashed paths leading toward it.
    1. 2.1The north star metric and its KPI treeOne north star metric for the value users get, and a KPI tree of input metrics that teams can actually move.~28 min

      2.1 The north star metric and its KPI tree

      ~28 min · 9 exercises

      StartRead theory
    2. 2.2Volume, quality and Simpson's paradoxTell how much from how well, and catch Simpson's paradox: a per-group ratio that rose only because the mix of groups changed.~30 min

      2.2 Volume, quality and Simpson's paradox

      ~30 min · 9 exercises

      StartRead theory
    3. 2.3DAU, WAU, MAU and stickinessCount active users at the rhythm people actually use the product.~31 min

      2.3 DAU, WAU, MAU and stickiness

      ~31 min · 9 exercises

      StartRead theory
    4. 2.4Case: a north star for HalvesPick and defend a north star metric for Halves, with six months of numbers behind it.~41 min

      2.4 Case: a north star for Halves

      ~41 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  3. Module 3: Where do new users get stuck?

    Funnels: order, conversion window and who counts; reading a funnel without fooling yourself; finding the activation moment.

    Small figures pour into a wide blue funnel above a phone with a split receipt, and a few get stuck at an orange narrow neck.
    1. 3.1Funnels: conversion window, step order and who countsHow a funnel conversion rate is computed, and why the conversion window, the step order and who enters give different numbers on the same users.~28 min

      3.1 Funnels: conversion window, step order and who counts

      ~28 min · 9 exercises

      StartRead theory
    2. 3.2Reading a funnel: right censoring and date rangesRight censoring (users who haven't had time to convert yet), date ranges and other ways a funnel conversion rate misleads.~31 min

      3.2 Reading a funnel: right censoring and date ranges

      ~31 min · 9 exercises

      StartRead theory
    3. 3.3Activation: finding the aha momentFind your product's aha moment: the week-one action that separates users who stay from users who leave, and know what that comparison can't prove.~30 min

      3.3 Activation: finding the aha moment

      ~30 min · 9 exercises

      StartRead theory
    4. 3.4Case: fix the onboarding funnelFind where new Halves users drop off and what to fix first.~42 min

      3.4 Case: fix the onboarding funnel

      ~42 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  4. Module 4: Do people come back?

    Day-N, rolling and bracket retention; reading and averaging a cohort table; the retention plateau as a product-market-fit signal.

    An orange boomerang curves away and back toward a phone with a split receipt.
    1. 4.1Retention: day N, rolling or bracketThree ways to measure retention and which fits a product people use weekly or monthly.~28 min

      4.1 Retention: day N, rolling or bracket

      ~28 min · 9 exercises

      StartRead theory
    2. 4.2Reading a cohort tableRead rows, columns and diagonals, and average cohorts without the classic mistakes.~28 min

      4.2 Reading a cohort table

      ~28 min · 9 exercises

      StartRead theory
    3. 4.3The plateau: retention and product-market fitWhat a flattening retention curve says about the product, and what it doesn't.~28 min

      4.3 The plateau: retention and product-market fit

      ~28 min · 9 exercises

      StartRead theory
    4. 4.4Case: retention analysis by segmentIs Halves retention healthy for each kind of group? Run a retention analysis by segment on a year of cohort data and write the verdict.~41 min

      4.4 Case: retention analysis by segment

      ~41 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  5. Module 5: Did the change work?

    Hypothesis, primary metric and a decision rule up front; test size and duration; who is in the test; reading a finished test honestly.

    Two versions of a phone with a split receipt on a balance scale, one with a blue screen and one with an orange screen.
    1. 5.1A/B test plan: hypothesis, metrics, guardrailsWrite the A/B test plan before launch: a hypothesis, one primary metric, guardrail metrics and a decision rule.~27 min

      5.1 A/B test plan: hypothesis, metrics, guardrails

      ~27 min · 9 exercises

      StartRead theory
    2. 5.2How big and how longSample size and duration for a product test.~31 min

      5.2 How big and how long

      ~31 min · 10 exercises

      StartRead theory
    3. 5.3Who is in the test: triggers and cluster randomizationWho belongs in an A/B test: triggered enrollment, dilution, and cluster randomization of whole groups when users affect each other.~29 min

      5.3 Who is in the test: triggers and cluster randomization

      ~29 min · 9 exercises

      StartRead theory
    4. 5.4Case: reading a finished testEffect, interval, guardrails and the decision.~44 min

      5.4 Case: reading a finished test

      ~44 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  6. Module 6: What can we learn without a test?

    Fair version comparisons; traffic mix and seasonality; correlation, selection and 'users who do X retain better'.

    A detective's magnifying glass between an older gray phone and a newer phone with a split receipt, orange footprints between them.
    1. 6.1Pre/post analysis: comparing versions fairlyPre/post analysis: the rules that make a before-and-after comparison of app versions worth reading.~30 min

      6.1 Pre/post analysis: comparing versions fairly

      ~30 min · 9 exercises

      StartRead theory
    2. 6.2Traffic mix and seasonalityWhat seasonality and a changing traffic mix do to product metrics, and how to compare like with like.~31 min

      6.2 Traffic mix and seasonality

      ~31 min · 9 exercises

      StartRead theory
    3. 6.3Selection bias: users who do X retain betterA selection bias example: correlation, self-selection, a confounding variable and what a feature's users can and can't tell you.~30 min

      6.3 Selection bias: users who do X retain better

      ~30 min · 9 exercises

      StartRead theory
    4. 6.4Case: the version or the traffic?Did the new Halves version help, or did the users change?~42 min

      6.4 Case: the version or the traffic?

      ~42 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  7. Module 7: Are we making money on it?

    ARPU, conversion to paid and trial-to-paid; the LTV curve by cohort; CAC, payback and ROI by cohort and channel; pricing and paywall decisions.

    Coins spill from the two halves of a receipt on a phone into a blue coin jar with an orange lid.
    1. 7.1How to calculate ARPU and ARPPUHow to calculate ARPU and ARPPU, conversion to paid and trial-to-paid, and how a change in the paywall trigger mix fakes a better paywall.~33 min

      7.1 How to calculate ARPU and ARPPU

      ~33 min · 9 exercises

      StartRead theory
    2. 7.2The LTV curveCumulative gross profit per cohort user by month, how to read it, and when a young cohort can be extrapolated.~30 min

      7.2 The LTV curve

      ~30 min · 9 exercises

      StartRead theory
    3. 7.3CAC, payback and ROIJudge a paid channel by the cohort it buys: CAC, payback period and cohort ROI at a stated age, and why a month's profit divided by that month's spend flatters ads.~31 min

      7.3 CAC, payback and ROI

      ~31 min · 9 exercises

      StartRead theory
    4. 7.4Price testing case: price or paywall?Price testing in practice: read a price test and a paywall before-and-after honestly, then recommend what Halves Plus should do next.~43 min

      7.4 Price testing case: price or paywall?

      ~43 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock

  8. Module 8: How will we grow?

    Forecasting active users from cohorts; measuring virality honestly; channels, capacity and modern attribution; investigating an unexplained spike.

    A young plant with blue leaves grows out of a phone with a split receipt, with one orange bud on top.
    1. 8.1Forecasting active usersNew users × retention: build a monthly active users forecast from cohorts, with a separate curve for summer cohorts, and price a cut in paid acquisition in users.~34 min

      8.1 Forecasting active users

      ~34 min · 9 exercises

      StartRead theory
    2. 8.2Invites and viral growthViral growth from invites: the viral coefficient k, cycle time, why k below 1 still pays, and how to measure invites without flattering yourself.~31 min

      8.2 Invites and viral growth

      ~31 min · 9 exercises

      StartRead theory
    3. 8.3Channels and marketing attributionChannel capacity and marginal CAC, a break-even cost that values the invitees a paid user brings, marketing attribution through the stores' privacy frameworks, and incrementality from a geo holdout.~33 min

      8.3 Channels and marketing attribution

      ~33 min · 9 exercises

      StartRead theory
    4. 8.4Capstone: the unexplained spikeSign-ups more than doubled over a weekend. List the possible causes, rule them out one by one with the data at hand, and write the memo.~43 min

      8.4 Capstone: the unexplained spike

      ~43 min · 10 exercises

      StartRead theory
    Module examComplete 4 more lessons to unlock