# A tracking plan for a new feature

> Before payment reminders ship, plan the data: the questions, the events and properties that answer them, where each is logged, the metric definitions and the checks before launch.

- Course: [Product Analytics: From Data to Decisions](https://productovi.com/courses/product-analytics.md)
- Module: What exactly are we measuring?
- Lesson 4/32, ~37 min
- Source: https://productovi.com/courses/product-analytics/tracking-plan

## Start from the questions

**Alex Tutor:** On the Halves product team, a new feature is on the way, and Maya wants your data plan before it ships.

**Maya:** Payment reminders ship in three weeks: Halves nudges whoever owes money, and they can open it and settle up. After launch I need two answers. Do reminders get people to settle faster? And do they annoy anyone?

**Sofia:** Then let's list the events: reminder sent, reminder opened, settle-up. Anything else?

**Alex Tutor:** Go one step back. A **tracking plan** is built from the questions down: first the questions, then the events and properties that answer them, then where each one is logged, then the metric definitions, and last a check before launch. An event nobody needs for a question is just noise.

![A speech bubble at the top branches downwards into small receipts and tags, ending at a clipboard with ticks.](https://productovi.com/courses/product-analytics/inline/tracking-plan-illustration-2.webp)

*First the questions, then the events that answer them, then the checks.*

**Alex Tutor:** Maya's two questions split into four. How many reminders reach people and get opened? How many reminded debts get settled, and how fast? Is that faster than without a reminder? And how many people mute reminders?

**Maya:** The last one worries me most. If a reminder spoils the mood in a flat, I want to know in week one, not in month three.

> **Exercise:** Put the steps of building a tracking plan for payment reminders in order.

## Events for each question

**Alex Tutor:** Now the events. The server sends a reminder, so `reminder_sent` is a **server-side event**: it's logged even when the push never arrives. As with recurring bills, it isn't user activity. `reminder_opened` is logged by the app when the person taps it.

**Alex Tutor:** For "how fast" you need a link. Give every reminder a `reminder_id`, and put it on `reminder_opened` and on `settle_up_recorded` when the settle-up starts from the reminder. Without that link, reminded settle-ups look like any other.

**Theo:** And annoyance? Nobody taps a button called "I'm annoyed".

**Alex Tutor:** No, but they act on it. Log `reminders_muted` when someone turns reminders off, with `scope`: this group or all groups. Mutes, and people who stop opening, are what annoyance looks like in data.

![A hand silences a ringing bell on a phone while flatmates in the background look annoyed at their own phones.](https://productovi.com/courses/product-analytics/inline/tracking-plan-illustration-3.webp)

*Nobody reports being annoyed, but turning reminders off is an action you can log.*

> **Exercise:** Match each question about payment reminders with the event that answers it.

**Alex Tutor:** Quick check on that server event before we move on. Theo already asked whether reminders could lift his active-user chart.

> **Exercise:** **True or false:** a user with a `reminder_sent` event this week can count as a weekly active user, because the reminder reached them.

## On the event or on the user?

**Sofia:** Where does "reminders are off" go? On the event or on the user?

**Alex Tutor:** Both. The mute is an event: it happened at a moment. Whether reminders are on right now describes the person, so it's also a **user property**, `reminders_enabled`. The debt amount and whether it's the first or second reminder describe one reminder: **event properties** on `reminder_sent`.

**Alex Tutor:** One rule: put on each event everything you'll later want to filter or group by. You can't add a property to last month's events.

**Maya:** Then I'll ask this on day one: do second reminders get muted more often than first ones?

> **Exercise:** Maya asks: "Do second reminders about the same debt get muted more often than first ones?" The draft plan has `reminder_sent` (`reminder_id`, `recipient_user_id`, `group_id`) and `reminders_muted` (`scope`). What must be added to answer her? Select all that apply.

**Alex Tutor:** Theo has a follow-up of his own: do bigger debts get muted more? Two ways to store the amount are on the table.

> **Exercise:** Where should the debt amount live, so Theo can compare mute rates for small and large debts?

## Definitions before data

**Alex Tutor:** Next, write each metric as a definition, like last lesson. Mute rate: of the **unique users** who got a reminder this week, the share who muted within 7 days. People, not mute events: someone toggling five times is one person.

**Sofia:** And "faster than without a reminder"? We could compare reminded debts with the others.

**Alex Tutor:** Careful: reminders go to debts that are already late, so the groups differ from the start. A fair comparison needs a **holdout group**: a random share of debts that would get a reminder but don't. Plan it now, with a `reminder_held_back` event.

![One reminder_id on the sent, opened and settled events measures time to settle; a held-back holdout gives a fair comparison, and mutes show annoyance.](https://productovi.com/courses/product-analytics/diagrams/m1-l4-reminder-events.png)

**Alex Tutor:** Now test the mute-rate definition on a made-up first week, before real data arrives.

> **Exercise:** Test week: 240 unique users got at least one reminder. Within 7 days, 15 of them muted reminders, logging 27 `reminders_muted` events between them (some toggled several times). Two other people, who got no reminder that week, muted from settings. Halves defines the mute rate as: of the unique users who got a reminder that week, the share who muted reminders within 7 days. What is the mute rate, in %?

## Check it before launch

**Alex Tutor:** Last step: QA. On a test account, trigger each event and look at the raw rows. Does it fire once, not twice on a retry? Are the properties filled, with no empty `reminder_id`?

**Alex Tutor:** Two more. Test accounts must be flagged as **internal traffic**, or QA taps become "opens". And times are logged in UTC, so keep the user's `timezone` to see when a reminder arrived locally.

**Theo:** That one matters. If reminders land at 3 a.m. in Bangkok, mutes jump and we'll blame the wording.

**Sofia:** So QA checks the data, not just whether the feature works.

![A person with a magnifying glass inspects a strip of receipts coming out of a test phone and flags one duplicate.](https://productovi.com/courses/product-analytics/inline/tracking-plan-illustration-4.webp)

*Before launch, QA checks the rows themselves, not only the feature.*

> **Exercise:** A developer's draft plan for reminders has one event, `reminder_shown`, logged by the app when the reminder appears on the phone. The open rate is computed as `reminder_opened` / `reminder_shown`. What is the main flaw?

**Alex Tutor:** Here's a real finding from yesterday's QA run on a test account. What would you do with it?

> **Exercise:** During QA, one reminder produced two `reminder_sent` rows with the same `reminder_id`, two seconds apart: the server retried after a timeout. What's the right response before launch?

**Maya:** QA is settled. Now write me the definition I care about most: time to settle.

> **Exercise:** Write the metric definition for **time to settle after a reminder** in 2–4 sentences. Say which events it starts and ends with, who is excluded, what happens to reminders that are never settled, and which number is reported.

## Your turn: trip budgets

**Maya:** Next quarter we add trip budgets. I'll want the same kind of plan for it, before anyone writes code.

**Alex Tutor:** Then run the same five steps: questions, events and properties with their source, one written definition, and the checks before launch.

**Sofia:** Starting from Maya's questions, not from the screens.

**Alex Tutor:** Exactly. Screens change; the questions are what the data is for.

> **Exercise:** Write a short tracking plan for Halves trip budgets.

> **Key ideas**
>
> - A **tracking plan** goes from questions down: questions → events and properties → where each is logged → metric definitions → QA before launch.
> - Log what the server does on the server (`reminder_sent`) and what the person does in the app (`reminder_opened`). Server events aren't user activity.
> - Link events with ids (`reminder_id` on the open and the settle-up), or you can't tell what the feature caused.
> - Event properties describe one moment; user properties describe the person. Add every property you'll filter by now: old events can't get it later.
> - Annoyance shows up as behavior: mutes and people who stop opening.
> - A fair "faster than without" needs a holdout group planned before launch.
> - QA checks the data: fires once, properties filled, internal accounts flagged, time zone kept.

**Alex Tutor:** That's the whole module: with events you trust, written definitions and a plan before launch, every number after it has a chance of being right.

- Previous lesson: [Metric definition: one rule, one number](https://productovi.com/courses/product-analytics/metric-definitions.md)
- Next lesson: [The north star metric and its KPI tree](https://productovi.com/courses/product-analytics/north-star.md)

---
Productovi Learning. Free, no sign-up needed to read.