# Capstone: the unexplained spike > Sign-ups more than doubled over a weekend. List the possible causes, rule them out one by one with the data at hand, and write the memo. - Course: [Product Analytics: From Data to Decisions](https://productovi.com/courses/product-analytics.md) - Module: How will we grow? - Lesson 32/32, ~43 min - Source: https://productovi.com/courses/product-analytics/spike-investigation ## Sign-ups more than doubled **Alex Tutor:** A long weekend is over at Halves, and Maya has a spike to celebrate. As always, the product team checks with you first. **Maya:** Monday morning, and the dashboard says sign-ups over the long weekend were up **156%**: 2,014 in three days. It's the new Android sign-up screen, isn't it? I want to tell the team. **Alex Tutor:** Maybe. Before anyone celebrates, list everything that could make a spike like this. Four families: **tracking**, **traffic**, **product** and **outside events**. **Sofia:** Tracking: we count something twice. Traffic: a campaign or a store placement. Product: a release changed what people do. Outside: a holiday, the news, a competitor's outage. **Alex Tutor:** Then rule them out one by one with the data we already have, cheapest check first. That's a **spike investigation**. Here's the change log for the period. ![A detective with a magnifying glass studies a sudden tall pile of receipts, with four closed doors behind them.](https://productovi.com/courses/product-analytics/inline/spike-investigation-illustration-2.webp) *A spike is a question: list the possible causes, then rule them out.* > **What changed, 29 August – 18 September** > > | Date | What changed | > |---|---| > | Thu 10 Sep | Android `5.2` starts rolling out: a new phone-number sign-up screen (about 30% of Android sign-ups that day, 70% on the 11th, all from the 12th) | > | Wed 16 Sep | Android `5.2.1` with bug fixes, from midday | > | Whole period | Paid social and store search ads budgets unchanged; no store featuring; no iOS release | > | Sat 12 – Mon 14 Sep | Public-holiday long weekend in several of Halves's biggest markets | > **Exercise:** Match each family of causes with the first check you'd run on this data. ## Up on what? **Alex Tutor:** "Up 156%" needs a baseline. Weekends and Mondays differ from midweek, so compare each spike day with the same weekday in the two weeks before, and average those two. **Sofia:** So Saturday the 12th against Saturdays the 29th and the 5th, and the same for Sunday and Monday. **Alex Tutor:** Yes. And count two things side by side: `signup_events`, which the dashboard shows, and `new_users`, distinct new accounts. **Dataset (CSV):** [Halves sign-ups by day, platform and channel](https://productovi.com/datasets/product-analytics/spike-investigation/halves-signups-september.csv) Saturday 29 August – Friday 18 September 2026; 21 days × 2 platforms × 4 channels. - `date`: Day (UTC), Saturday 29 August – Friday 18 September 2026 - `platform`: ios or android - `acquisition_channel`: invite, app_store_search, paid_social or store_search_ads - `signup_events`: Number of `signed_up` events: what the dashboard calls "sign-ups" - `new_users`: Distinct new accounts created that day - `new_users_in_trip_groups`: Of the new users, those whose first group is a trip group **Alex Tutor:** The baseline for the three days is 785.5 in both columns. Now compare the spike days with it. > **Exercise:** By how much did **distinct new users** on 12–14 September exceed the baseline (the average of the same weekdays in the two previous weeks)? Give the change in percent. **Alex Tutor:** Why bother with weekdays two weeks back? Here are two baselines someone could have picked. > **Exercise:** Which baseline is **fair** for the spike on Saturday 12 – Monday 14 September? ## Tracking first **Sofia:** Events went up 156% but new accounts only 77%. So 620 events on those days aren't new people. **Alex Tutor:** Split by platform. On iOS it's 1.00 event per new user, as always. On Android, 2.00 on all three days. **Sofia:** And the Android extras start on the 10th, the day `5.2` began rolling out, and stop after the fix on the 16th. **Alex Tutor:** The new screen sends `signed_up` twice. That's half of Maya's excess: a counting bug, not people. ![One person signs up on a phone that prints out two identical receipts instead of one.](https://productovi.com/courses/product-analytics/inline/spike-investigation-illustration-3.webp) *Half the jump was a counting bug: one screen sent each sign-up twice.* > **Exercise:** Halves's dashboard counts sign-ups as `signed_up` events. Android `5.2`, with a new sign-up screen, started rolling out on 10 September. On 12–14 September Android shows 2.00 `signed_up` events per new user and iOS 1.00; Android's extra events start on 10 September and stop after 16 September. What follows? **Alex Tutor:** One more test makes the diagnosis airtight: the rollout went 30%, 70%, then 100%. Do the duplicates follow it? > **Exercise:** Use the sign-up table. On **Friday 11 September**, when `5.2` reached about 70% of Android sign-ups, how many `signup_events` were there per distinct new user **on Android**? Round to two decimals. ## Traffic and product **Maya:** Fine, but 608 real extra users is still a lot. Where did they come from? **Alex Tutor:** By channel: invites +555.5, store search +57.5, paid social −2, store search ads −2.5. No campaign did this, and budgets didn't change. **Sofia:** And by platform: iOS new users +77.7%, Android +77.1%. If the Android screen had helped, Android would rise more. **Alex Tutor:** So the release explains the extra events, not the extra people. **Maya:** So the new screen did nothing at all? **Alex Tutor:** Nothing we can see in sign-ups. Whether it's easier to use is a separate question, and an A/B test, not a spike, can answer it. > **Exercise:** Which explanations of the **real** rise in new users can you rule out with this data? Select all that apply. **Alex Tutor:** Put a number on the channel split before we look outside the product. > **Exercise:** Distinct new users on 12–14 September exceeded the baseline by 608.5. Invites brought 902 new users on the spike days against a baseline of 346.5. What share of the **excess** came through invites? Give a percentage. ## Something outside **Theo:** The 12th to the 14th was a long holiday weekend in several of our biggest markets. People go away in groups. **Alex Tutor:** And the data agrees: 47% of spike-day new users started in a trip group, against 10% on the baseline days. That's consistent with the holiday, though a comparison like this can't prove it. ![Of 2,014 sign-up events, 620 were Android duplicates and 608 real extra users, mostly invites into trip groups on a holiday weekend; traffic and product ruled out.](https://productovi.com/courses/product-analytics/diagrams/m8-l4-spike-breakdown.png) ![Friends with suitcases gather around one phone at a station, setting off for a long holiday weekend.](https://productovi.com/courses/product-analytics/inline/spike-investigation-illustration-4.webp) *The real extra users came with a holiday weekend, not a campaign or a release.* **Sofia:** Then these users may behave like the summer cohorts and leave after the trip. **Alex Tutor:** First, one check: how strongly can we state the holiday explanation? > **Exercise:** Because 47% of spike-day new users started in a trip group (against 10% on baseline days) and the dates match a holiday weekend, the data proves the holiday caused the rise in new users. **Alex Tutor:** Right, so nobody should raise the autumn forecast on this weekend. And the dashboard needs fixing so the next spike isn't half fake. > **Exercise:** Halves's dashboard counts sign-ups as `signed_up` events. Android `5.2`, rolled out from 10 September, logged each sign-up twice, so the dashboard showed +156% for the 12–14 September weekend while distinct new users rose 77%. Fix the dashboard in 3–5 sentences: rewrite the definition of "sign-ups" so this can't happen again, and describe one automatic check that would have caught the Android bug on 10 September. ## The memo **Maya:** OK. I need something I can forward to the team today, before the rumor about the new screen spreads. **Alex Tutor:** Lead with the answer, then the checks, with numbers: what was ruled out, what's left, how sure we are, and what we're fixing. > **Exercise:** Write the memo to Maya: what caused the sign-up spike on 12–14 September? **Alex Tutor:** The habit to keep: a spike is a question, not a result. List the families, compare with a baseline, check tracking first, and say how sure you are. - Previous lesson: [Channels and marketing attribution](https://productovi.com/courses/product-analytics/channels.md) --- Productovi Learning. Free, no sign-up needed to read.