Case: retention analysis by segment

A retention analysis by segment starts with two decisions, made before you look at any result.

1. The measure. Here it's monthly activity by app opens: a user is retained in month k if they opened Halves at least once that month. "Any event" stays in the table only as a check, because it also counts recurring bills the server creates on its own.
2. The segment. Split by whatever sets the usage rhythm. For Halves that's the type of the user's first group: couple, flat or trip.

Then read each segment's rows on their own. For couples, month 12 is 316 of 600, so 316 ÷ 600 = 52.7%. Month 6 was 330 ÷ 600 = 55.0%, so the curve has been nearly flat for half a year: a plateau.

The last column adds a useful detail: only 12 of the 316 active couples in month 12 are in a group created after sign-up. Couples keep the group they started with, which is what steady, ongoing use looks like.

Always divide by the segment's own cohort size, not by all 3,000 summer sign-ups.

To judge whether a curve has really stopped falling, measure how much it moved over the stretch you call a plateau.

Couples: 55.0% in month 6, 52.7% in month 12. The change is 55.0 − 52.7 = 2.3 percentage points over six months, less than half a point a month. For comparison, the same curve lost 72.2 − 64.5 = 7.7 points between month 1 and month 2 alone.

Keep the units straight. A difference between two percentages is in percentage points (pp). The same change can also be expressed in relative terms: 2.3 ÷ 55.0 ≈ 4.2% of the month-6 level. Both are correct, but they are different numbers, and mixing them up causes real confusion in reports. "Retention fell by 4%" could mean 55.0% → 52.7% or 55.0% → 51.0%.

A practical rule: say "points" for differences between rates, and write "relative" when you mean a percentage change of a rate. For plateau checks, points per month is the clearest yardstick: a curve losing under about half a point a month for several months is behaving like a plateau.

The difference between two activity counts is a number of its own, but subtracting counts is valid only when one group contains the other.

Flats, month 12: 403 users had any event, 315 opened the app. Everyone who opened the app also had an event, so 403 − 315 = 88 users were active only through the server, mostly recurring rent bills (the server-created activity from The plateau: retention and product-market fit). That's 88 ÷ 800 = 11% of the cohort; counted, they would lift flats from 315 ÷ 800 = 39.4% to 403 ÷ 800 = 50.4%.

For couples the same gap is small (328 − 316 = 12), and for trips it's zero, because trip groups rarely have recurring bills. A gap that is large in one segment and tiny in others shows exactly where automated activity sits.

If each column could include people missing from the other, the difference of two counts would mean nothing. You'd need the user lists and their overlap, not just two numbers.

In a memo, server-only activity is an open question, not retention, and it takes three lines: the headline, the gap, and how you'll resolve it.

Both easy answers fail for the 88 flat users. Counting them as retained claims what the table can't show, that they still get value from Halves. Dropping them silently loses a real signal: something in flats keeps running without users.

The three lines:

1. Headline by the user measure: flats are at 39.4% in month 12 by app opens.
2. Name the gap: 88 users (11% of the cohort) are active only through server events; by any event the figure would be 50.4%.
3. Say how you'd resolve it: do those users still record settle-ups or edit the bill? Did their group stop adding other expenses months ago, which would suggest someone moved out and nobody switched the bill off?

Automation can be genuine value: a flat whose rent splits itself may be perfectly happy. But that's a hypothesis until a user action backs it, so the 88 stay a question in the memo.

When someone calls a seasonal segment churn, check where its returning users come back: to the old group or to a new one. Trips fall to 89 of 1,600 users (5.6%) by month 5, and the CEO calls that churn. Yet by month 12, 349 users (21.8%) are active again.

The column active_in_new_group says who they are: 304 of the 349 are in a group created after they signed up, 304 ÷ 349 = 87.1%. They didn't return to last summer's trip group; they started a new trip.

Compare the other segments in month 12: flats 40 ÷ 315 = 12.7%, couples 12 ÷ 316 = 3.8%. Couples and flats come back to the same group; trip users come back with a new one.

This is the difference between group lifetime and user churn from The plateau: retention and product-market fit, now backed by data: the trip group ended, the people didn't leave. Counting every ended trip group as a lost user would make the most seasonal segment look like a failure even when its users return every year.

The 304 trip users who are active in a new group in month 12 can be turned into two different shares, and both are useful as long as you say which one you mean.

• Share of the active users: 304 ÷ 349 = 87.1%. It answers "why are the people who came back here?" Almost all of them came for a new trip.
• Share of the whole cohort: 304 ÷ 1,600 = 19.0%. It answers "what share of last summer's trip users came back for another trip within a year?" That's the number to track for trips, because it compares directly with the size of the campaign that brought them in.

The two sound similar and differ by a factor of more than four. Putting "87% of trip users come back for a new trip" in a memo would be a serious overstatement: 87% is the share among the 22% who came back at all.

Before writing any percentage, name its denominator in words ("of the cohort", "of active users", "of trip groups"). If the sentence still sounds right with the denominator spelled out, the number is being used honestly.

A segment's retention metric needs a window at least as long as the segment's natural usage rhythm. People take one or two trips a year, so a trip metric has to look across a year.

A good candidate: the share of trip users who start or join a new trip group within 12 months of sign-up. For last summer's cohort that is 304 ÷ 1,600 = 19.0%. It rewards exactly what a travel product should do: be the app people reach for again on the next trip.

Why the alternatives don't work:

• Monthly activity calls the quiet months between trips churn; it's built for products with a monthly rhythm.
• Activity in month 3 lands in the gap between summer trips for almost everyone, so it mostly measures timing.
• Expenses per trip group measures how busy one trip was, not whether anyone comes back.

The same check works for any segment: write down how often a satisfied user would naturally return, then choose a window at least that long. Couples and flats fit a month; trips need a year.

Give each segment its own verdict: a single number for segments with different rhythms answers no one's question, as The plateau: retention and product-market fit showed for curves. Here all 3,000 summer sign-ups give 980 ÷ 3,000 = 32.7% active in month 12, a blend of a steady plateau, automated activity and a seasonal return, weighted by last summer's mix.

A memo that helps a budget decision gives each segment one line:

• Couples: 52.7% in month 12, flat since month 6. Healthy.
• Flats: 39.4% by app opens; 88 users (11%) active only through server bills, to be checked.
• Trips: 5.6% in month 5 by design; 21.8% back by month 12, 19.0% of the cohort in a new trip group.

Each line names its measure, so the CEO's "churn" and Theo's "double it" can be tested against the right row.

How it fits together: fix the measure first, read each segment by its own rhythm, keep server-only activity as an open question, and name units and denominators. Module 4 answered whether people come back; the next module asks whether a change worked.