# Traffic mix and seasonality

> What seasonality and a changing traffic mix do to product metrics, and how to compare like with like.

- Course: [Product Analytics: From Data to Decisions](https://productovi.com/courses/product-analytics.md)
- Module: What can we learn without a test?
- Lesson 22/32, ~31 min
- Source: https://productovi.com/courses/product-analytics/traffic-and-seasons

## A retention drop after a summer release

**Alex Tutor:** Another release, another drop at Halves, this time at the start of the summer season. The team wants to roll back, and Maya wants your data first.

**Maya:** Version 5.0.0 went out on 6 July, and week-2 retention of new users has slid from **37.0% in June to 30.3% in July**. The team wants the old home screen back.

**Sofia:** Before we blame the screen: what else was different in July?

**Alex Tutor:** Good first question. The metric is identical in both months: new users, days 8–14, every cohort complete. What can differ is the crowd inside it. And July is when Halves users go on holiday together.

**Alex Tutor:** Here are the weekly cohorts, split by channel and by the type of group each person joined first. Start with the season: how much of July arrived through trip groups?

**Dataset (CSV):** [Weekly sign-up cohorts, June and July](https://productovi.com/datasets/product-analytics/traffic-and-seasons/halves-summer-cohorts.csv)

New Halves users, weeks of 8 June to 27 July 2026. Version 5.0.0 was released on 6 July. Every cohort has its full days 8–14.

- `signup_week`: Monday of the sign-up week
- `app_version`: Version installed: 4.9.0 in June, 5.0.0 in July
- `channel`: invite, app_store_search, paid_social or partner_promo
- `first_group_type`: Type of the first group the user joined: flat, couple or trip
- `new_users`: New users in this cell
- `active_days_8_14`: Of them, opened the app on at least one of days 8–14

> **Exercise:** What share of **July (5.0.0)** new users had `trip` as their first group type? Percent, two decimals. (In June it was 20.02%.)

## The trip season: who arrives in July

**Alex Tutor:** Almost double June. A trip group comes with an end date: the holiday is over, the last dinner is split, everyone settles up. Days 8–14 often fall after the flight home, so a trip user who stays away may be perfectly satisfied.

![A summer wave of travelers with suitcases and sun hats arrives, outnumbering the flatmates and couples among the newcomers.](https://productovi.com/courses/product-analytics/inline/traffic-and-seasons-illustration-2.webp)

*July brings many more trip groups, and they end when the holiday does.*

**Sofia:** So even if 5.0.0 changed nothing, more trips would drag the July average down.

**Alex Tutor:** Yes. The **traffic mix** moved, and a **mix shift** moves the total while no segment changes. In June trip users kept 16.5% in days 8–14 and flats 43.3%. Trade flats for trips and the average sinks by itself.

**Theo:** There’s a second thing, and it’s mine. For two weeks from 13 July a bank’s student card paid holders a $5 bonus for signing up to Halves and creating a group. Those are the `partner_promo` rows.

**Alex Tutor:** Then those are **incentivized sign-ups**: people who joined for the reward. Some really had rent to split; many made a group only to meet the condition and never added a bill. How big was the wave?

> **Exercise:** What share of the new users who signed up on **5.0.0** (July) came through `partner_promo`? Percent, two decimals.

## Like-for-like comparison

**Alex Tutor:** Over a quarter of July, and only **19.4%** of them were active in days 8–14, below every regular channel in June (29–41%). June had no such channel, so they have no counterpart: set those rows aside first. July without them: **34.24%**, still under June’s 36.99%.

**Maya:** Still lower than June, though.

**Alex Tutor:** Because the extra trips are still in there. Step two: compare inside each group type, trips against trips, flats against flats. Try flats.

![At a market stall a person weighs apples against apples and pears against pears, each kind in its own crate.](https://productovi.com/courses/product-analytics/inline/traffic-and-seasons-illustration-3.webp)

*Compare trips with trips and flats with flats before judging the release.*

> **Exercise:** Without the `partner_promo` rows, what share of **July `flat`** users was active in days 8–14? Percent, two decimals. (June flats: 43.25%.)

## Mix adjustment: one number for the whole picture

**Alex Tutor:** Flats, couples and trips each did slightly better in July. For one summary number, keep June’s mix and swap in July’s rates: weight each channel × group type cell of July by that cell’s share of June.

$$
r^{\text{July}}_{\text{adj}} = \sum_{c} w^{\text{June}}_{c}\, r^{\text{July}}_{c}
$$

*July’s retention at June’s mix: w_c is cell c’s share of June’s new users, r_c is July’s retention in that cell, promo rows left out. Here 38.89%, against 36.99% in June.*

**Sofia:** So at June’s traffic, July would be 1.9 points **better**, not 6.7 worse.

**Maya:** Then 5.0.0 improved retention. Great, I’ll say that.

## Seasonality: what it does to product metrics

**Sofia:** So every July the same dip comes back, release or no release?

**Alex Tutor:** Very likely. That’s **seasonality**: a pattern that repeats with the calendar. The main thing summer changes here isn’t how each kind of group behaves; it’s who arrives. Among regular sign-ups, without the promo, trips went from 20.0% in June to 40.0% in July.

**Alex Tutor:** Try putting a number on the season yourself: June's rate for each group type, weighted by July's mix.

> **Exercise:** Halves's week-2 retention of new users was 37.0% in June. Keep June's rates by group type (flats **43.25%**, couples **40.29%**, trips **16.46%**) and weight them by July's regular mix, bank-promo sign-ups left out (flats **35.9%**, couples **24.1%**, trips **40.0%**). What July rate does the season alone predict? Percent, one decimal.

**Alex Tutor:** You can put a number on it. Keep June’s retention for each group type (flats 43.25%, couples 40.29%, trips 16.46%) and weight it by July’s regular mix: 35.9%, 24.1%, 40.0%. That gives **31.8%**: the season alone would pull June’s 37.0% down by about five points.

**Alex Tutor:** That 31.8% is the seasonal baseline, and July’s regular users beat it with 34.2%. To adjust for seasonality: hold the mix fixed, read each segment on its own, and compare with the same weeks last year. Never read June against July raw.

**Maya:** Next July I'll want a quick check on day one. What do I compare against?

> **Exercise:** At Halves, summer brings many trip groups, which go quiet once the holiday ends. Next summer, Maya, the product lead, wants to check a July release quickly. Which comparison is **less** distorted by the season?

![July's raw 30.3% hides a promo wave and more trips; without the promo it is 34.24%, above the 31.8% seasonal baseline, and 38.89% at June's mix.](https://productovi.com/courses/product-analytics/diagrams/m6-l2-like-for-like.png)

![A circle of the seasons, sun, falling leaves, snowflakes and blossom, with a suitcase resting on the summer side.](https://productovi.com/courses/product-analytics/inline/traffic-and-seasons-illustration-4.webp)

*The same summer dip comes back every year, release or no release.*

## What adjustment can and can’t do

**Sofia:** So a mix-adjusted number is a fair comparison?

**Alex Tutor:** Fairer, not fair. It removes the differences you measured: channel and group type. Anything you didn't split by, like the weather or the school holidays, stays inside.

**Theo:** The bank wants to rerun the offer when the university term starts. What should I do differently?

**Alex Tutor:** First, settle how the July offer gets judged at all.

> **Exercise:** For two weeks in July a bank paid its student-card holders \$5 to sign up to Halves and create a group: 788 people joined that way, 26.69% of July's sign-ups, and 153 of them were active on days 8–14. Theo, the Halves growth marketer, has to tell the bank whether the offer worked. Which number should the deal be judged by?

**Alex Tutor:** Give the offer its own sign-up code, so every bonus user carries `partner_promo` from day one. Judge the deal by users still active in week 2, not by sign-ups: 788 joined, 153 stayed. And agree the dates with Maya, so a release and a promo don’t start in the same fortnight.

> **Exercise:** Halves's July week-2 retention of new users, with the bank-promo sign-ups set aside and re-weighted to June’s mix of channel × group type, is 38.89% (June: 36.99%). Which of these can still be hiding inside that number? Select all that apply.

**Alex Tutor:** Back to “5.0.0 improved retention”, Maya: it adjusts only for what we split by. July also brought holidays and heat, and some cells hold little more than a hundred users. It supports “no sign 5.0.0 hurt retention”, not “5.0.0 added 1.9 points”.

> **Exercise:** Halves shipped version 5.0.0, with a new home screen, on 6 July, and week-2 retention of new users fell from 37.0% in June to 30.3% in July. Maya, the product lead, has to answer the team today. Which statement can she defend with the weekly cohort table?

**Maya:** Fine, no rollback. But I need to explain it to the team without a lecture.

**Alex Tutor:** Then keep it to three moves: who came in July and how they differ, what the like-for-like numbers show, and what we can and can’t say.

**Sofia:** With a number behind each move, so nobody has to take our word for it.

> **Exercise:** Halves's week-2 retention of new users fell from 37.0% in June to 30.3% in July, after version 5.0.0 with a new home screen shipped on 6 July. Maya, the product lead, asks in the team channel: “Do we roll back 5.0.0?” Reply in 3–5 sentences: give the answer first, then the numbers behind it, then what the data can’t tell us.

- Previous lesson: [Pre/post analysis: comparing versions fairly](https://productovi.com/courses/product-analytics/version-comparison.md)
- Next lesson: [Selection bias: users who do X retain better](https://productovi.com/courses/product-analytics/correlation-and-selection.md)

---
Productovi Learning. Free, no sign-up needed to read.