# What would have happened anyway > A campaign "brought 410 families". Separate the families who come back every year, compare with last year and with holdout cities, and compute the return on contribution and its range. - Course: [Marketing Strategy: Who to Win and How](https://productovi.com/courses/marketing-strategy.md) - Module: Did it work, and what's the plan? - Lesson 30/32, ~28 min - Source: https://productovi.com/courses/marketing-strategy/what-would-have-happened-anyway ## 410 families for 150,000 **Alex Tutor:** Back at Klasna's marketing team: the metrics row needs one honest example, and the September restart campaign has just finished. **Uliana:** From 20 August to 30 September we spent 150,000 on the restart campaign in nine cities, and it brought 410 families. That's under 366 a family! **Leo:** Last October a new family cost us 4,000. This is ten times cheaper. Let's double it next year. **Hanna:** Before anyone doubles anything: four similar cities were left without the campaign on purpose. Let's use them. **Sofia:** How were the four holdout cities chosen? **Hanna:** Before launch, by size and by how many new families each had last September, so they looked like the campaign cities. Picking them after the results would have bent the test. **Alex Tutor:** Hold on to that point about timing. Quick check: > **Exercise:** Choosing the holdout cities after the campaign is fine, as long as they're similar in size to the campaign cities. **Dataset (CSV):** [September 2026 restart campaign by city](https://productovi.com/datasets/marketing-strategy/what-would-have-happened-anyway/klasna-september-restart-2026.csv) Thirteen Ukrainian cities: nine got the campaign (20 August – 30 September 2026), four similar cities were chosen as holdouts before launch. Klasna ran no restart campaign in September 2025. EU families were not part of the test. - `city`: City - `group`: campaign or holdout (no campaign) - `spring_families_2026`: Families who studied in February–May 2026 - `new_sep_2025`: New families with a first lesson in September 2025 - `new_sep_2026`: New families with a first lesson in September 2026 - `returning_sep_2026`: Spring families who came back in September 2026 - `campaign_spend_uah`: Campaign spend in the city, UAH - `answered_ad_sep_2026`: September 2026 families who answered "an ad" to "how did you hear about us?" **Alex Tutor:** Start by splitting the 410. In the campaign cities, how many are new families and how many are spring families coming back? > **Exercise:** In the campaign cities, what **share of the 410 families** were spring families coming back? In %, to two decimals. ## The families who come back anyway **Alex Tutor:** Every September about 30% of spring families come back to Klasna. That's the baseline for returners. Compare the return rate in the campaign cities with the holdout cities. **Sofia:** Campaign cities: 135 of 450, which is 30%. Holdout cities: 45 of 150, also 30%. **Alex Tutor:** So the campaign added nothing on returners. They'd have come back with or without the ads, and the reminder they needed costs nothing in a message to families who agreed to hear from Klasna. > **Exercise:** What did the campaign add to the number of **returning** spring families? ## New families against a baseline **Uliana:** Fine, but new families really grew: 230 last September, 275 this one. That's 45 more. **Alex Tutor:** Last year alone is a baseline, but the market moved too. In the holdout cities new families also grew, from 80 to 88, with no campaign at all. Take that growth off before you credit the ads. > **Exercise:** The campaign cities had 230 new families in September 2025 (no campaign then) and 275 in September 2026. The holdout cities, with no campaign, grew new families from 80 to 88. Applying the holdout growth to the campaign cities' 230, how many **new families did the campaign add**? ## Return on contribution **Hanna:** Now the money. A new family leaves about 8,170 of contribution in its first twelve months; that's our planning curve. Revenue doesn't count here, only what Klasna keeps. **Alex Tutor:** Return on contribution is the extra contribution minus the cost, divided by the cost. Work it out for the 22. > **Exercise:** 22 incremental families × 8,170 UAH of contribution each, against a campaign cost of 150,000 UAH. What is the **return on contribution**? In %, to two decimals. **Alex Tutor:** And the cost per family, on the honest count? > **Exercise:** The restart campaign cost **150,000 UAH** and added **22** new families. What is the cost per incremental family? In UAH, to two decimals. **Sofia:** Using last year alone, it would have been 45 families and 145.1%. **Alex Tutor:** And with no baseline at all, 410 families. Same campaign, three answers. Also, the four holdout cities grew between 9.09% and 11.11% each, so the true figure is somewhere around 19 to 24 families, a return of about 6% to 31%. **Alex Tutor:** So how would you state that in the report? > **Exercise:** The four holdout cities grew new families by 9.09% to 11.11% each. How should the report state the campaign's result? ## What the sign-up question says **Uliana:** But 168 families in the campaign cities said they found us through an ad! **Alex Tutor:** And 14 in the holdout cities said the same, where the campaign never ran. People remember the last thing they saw, and returners saw the ads too. Attribution answers are clues; the holdout is the evidence. > **Exercise:** Which conclusions does the September data support? Select all that apply. **Leo:** So do we double it or not? **Alex Tutor:** Write Leo the answer as you'd put it in the plan: what the campaign added, what it returned, and what changes next September. > **Exercise:** The September restart campaign cost 150,000 UAH and "brought 410 families": 135 were spring families returning at the usual 30% rate (the same in the holdout cities), and against the holdout growth it added about 22 new families (roughly 19–24). Each new family leaves about 8,170 UAH of contribution in twelve months. Leo wants to double the budget. Answer Leo (3–4 sentences): what the restart campaign really added, its return on contribution, and what you'd change next September. > **Key ideas** > > - A campaign's result is the difference from what would have happened anyway, not the total that showed up. > - Take off the customers who return every season (here 30% of spring families, with or without ads), then the growth seen in holdout cities. > - Here: 410 claimed, 22 incremental; 179,740 UAH of contribution against 150,000 of cost, a return on contribution of 19.83% (about 6–31% across the holdout cities). > - Never compute the return on revenue or on all customers in the period. Use incremental contribution. > - "How did you hear about us?" and other attribution reports are clues. Holdouts and baselines are the evidence; say how uncertain they are. **Alex Tutor:** Next lesson: once you can measure, you need a habit of looking every month and acting on what you see. - Previous lesson: [From activity to profit](https://productovi.com/courses/marketing-strategy/from-activity-to-profit.md) - Next lesson: [The monthly control loop](https://productovi.com/courses/marketing-strategy/the-monthly-control-loop.md) --- Productovi Learning. Free, no sign-up needed to read.