Models and forecasting

A mathematical model is a simplified, formal description of something real. It keeps only the parts that matter for a question, so you can explore "what if" without running the experiment on real customers: what if delivery got a day faster, what if the price rose by 10%.

Models differ in what they connect.

• A functional model links inputs to one target. "Churn next month depends on price, delivery speed and support tickets" written as an equation is functional: you feed in the inputs and get a prediction. Regression models are functional.
• A structural model breaks one concept into its components. "Customer loyalty = trust + habit + willingness to recommend" doesn't predict loyalty from anything else; it describes what loyalty is made of. Factor analysis, later in this module, builds this kind.

Before any equation there is a content model: a plain-language list of what might matter and why. It decides which data you collect. Skip it and you'll fit a beautiful formula to whatever columns happened to be in the database, which is how important drivers get left out.

Models also differ in the kind of values their variables take, and that decides which tools fit.

• A continuous variable can take any value within a range: order value (412.37 UAH), parcel weight (2.41 kg), delivery time (26.5 hours). Between any two values there is always another possible one. Averages, standard deviations and regression lines suit it.
• A discrete variable takes a limited set of separate values: box size (small, medium, large), plan tier, number of items in an order. You count or label it rather than measure it.

In practice the line is sometimes drawn by usefulness rather than strict theory. Money is technically counted in kopiykas, but with thousands of possible amounts it behaves like a continuous variable. A rating from 1 to 5 is discrete and ordered.

Why it matters: a box size coded as 1, 2, 3 is still a label. Averaging it ("the mean box is 1.8") or fitting a slope to it as if "large" were exactly three times "small" gives numbers that look precise and mean little. Discrete labels enter a model as 0/1 indicator variables instead.

Describing a model takes two independent questions, and each has two answers.

What does it relate?

• functional: inputs → one target, through an equation (churn from price and support speed);
• structural: one concept → its parts (loyalty as trust, habit and advocacy).

Is time in the picture?

• static: one moment, a snapshot (this week's mix of customer segments);
• dynamic: change over time (orders growing by 15 a week, a subscriber base shrinking by churn each month).

The two questions combine freely. A forecast of weekly orders from ad spend is functional and dynamic. This quarter's breakdown of NPS drivers is structural and static. Most real business models are mixtures: a dynamic revenue forecast can include continuous revenue alongside discrete plan tiers.

The labels aren't trivia: they tell you which questions a model can answer. A static snapshot can't forecast next month, however good it looks. A structural breakdown can't tell you what will happen if you change the price. Before trusting a model's answer, check that its type matches the question.

The simplest forecast fits a straight line to a time series, a metric recorded at regular intervals, and extends it.

CatChow's weekly orders for weeks 1–4 were 200, 215, 230 and 245. They rise by exactly 15 a week, so the fitted line is

y = 185 + 15 × week

(185 is the intercept: the line's value at the imaginary "week 0"). To forecast, plug in the week number:

• week 5: 185 + 15 × 5 = 260;
• week 6: 185 + 15 × 6 = 185 + 90 = 275.

Two habits keep the arithmetic right: multiply before you add (185 + 15 = 200, then × 6 = 1,200 is a classic spreadsheet slip), and use the week number, not the number of weeks ahead.

The bigger question is how far to trust it. A line only knows the weeks it was fitted on. One or two steps ahead, it's a reasonable guess. Further out, real trends bend: seasonality (holiday peaks, summer dips), market saturation, a competitor's promotion. Four weeks of data can't support a five-year plan, however straight they look.

Once the real number arrives, check the forecast. The residual is the miss for one period:

residual = actual − predicted

Week 5 came in at 258 against a forecast of 260, so the residual is 258 − 260 = −2. Negative means the model overestimated; positive means it underestimated. Keep the order fixed; flipping it flips every sign and your story about which weeks beat the plan.

A miss of 2 sounds small, but small compared with what? Always compare a model with a naive baseline: the laziest forecast that needs no model at all. For weekly data that's usually "same as last week". Here it would have said 245 for week 5 and missed by 258 − 245 = 13, more than six times the trend model's miss.

A model that can't beat the naive baseline isn't adding anything, however sophisticated it looks. And one good week proves little: judge a model on its misses over several periods it hasn't seen.

Every dataset contains a pattern and some noise: one-off events, random wobble. A model with enough flexibility (ten variables, a wiggly curve) can pass through every past point exactly. That isn't skill. It has learned the noise too, and noise doesn't repeat. This is overfitting: excellent on the past, poor on the future.

The defence is a train/test split (holdout). For time data:

1. Hide the most recent weeks, say the last quarter of the data.
2. Fit the model on the earlier weeks only.
3. Forecast the hidden weeks and measure the misses.

The error on the hidden weeks is your honest estimate of forecasting skill. If a model fits the training weeks perfectly but misses the hidden ones badly, it has overfitted; a simpler model that does better on the hidden weeks wins.

That's also why a high R² on historical data is no reason to brag at stand-up. It measures how well the model describes data it was built from, not how well it will predict.

A forecast is an estimate, and every estimate comes with uncertainty. Writing it as a single number hides that: "275 orders" reads like a promise, and when week 6 lands at 268 the forecast looks "wrong" even though it did its job.

Write a forecast as a central value plus a range: "about 275, most likely between 260 and 290" or "275 ± 15". Where does the range come from? The simplest honest source is the model's own past misses on data it didn't see. If the trend model has typically been off by 5–10 orders on held-out weeks, a range of ±15 is a sensible cushion; a range of ±1 would be wishful thinking.

The range should grow the further ahead you forecast: next week is less uncertain than week 12, because small errors in the slope pile up.

An interval changes decisions. Warehouse staff plan stock for the top of the range, finance plans cash for the bottom, and nobody is surprised when the actual number lands anywhere inside. It also keeps the conversation honest when the real number wobbles.

Four habits separate a forecast you can act on from a nice-looking chart.

1. Test on unseen data. Judge the model by its misses on a holdout period, not by how well it fits the weeks it was built on.
2. Keep it simple until the data demands more. Each extra variable must earn its place on the holdout, not just raise the fit on the past. Adding every available column is the quickest route to overfitting.
3. State an interval. A central value with a range tells readers how much to trust it and protects you when reality lands slightly off.
4. Re-check when the context changes. A model learns the world as it was: old prices, last year's seasons, the competitors who existed then. A new pricing scheme, a new sales channel or a new competitor can quietly break it, so re-test after every such shift.

Two tempting moves to resist: picking the model with the lowest error on its own training data, and ignoring new data because the old model "worked before". Both feel rigorous and both lead to stale, overconfident forecasts.

One residual says little; you need a score over several periods the model never saw. The simplest is the mean absolute error (MAE): take each miss, drop its sign, and average.

For the trend model y = 185 + 15 × week, checked on weeks 5–7:

• forecasts 260, 275, 290; actuals 258, 268, 284;
• misses |−2| = 2, |−7| = 7, |−6| = 6;
• MAE = (2 + 7 + 6) / 3 = 15 / 3 = 5 orders.

Why drop the sign? Without it, misses in opposite directions cancel: a model that's 50 too high one week and 50 too low the next would average 0, a "perfect" score for a terrible forecast.

MAE is in the metric's own units, so it reads naturally: "typically off by about 5 orders a week". It's most useful as a comparison. The naive "same as last week" model forecasts 245, 258, 268 for the same weeks and misses by 13, 10 and 16, an MAE of 13. The trend model is clearly better.

Notice also that every residual here is negative: the trend runs a little ahead of reality. A consistent sign is an early hint that growth is slowing.