# Multiple linear regression > Multiple linear regression: separating the effect of several drivers at once, and the two classic traps, overfitting and multicollinearity. - Course: [Statistics for Product Managers and Marketers](https://productovi.com/courses/statistics-for-pm-and-marketers.md) - Module: Relationships - Lesson 19/24, ~39 min - Source: https://productovi.com/courses/statistics-for-pm-and-marketers/multiple-regression ## Multiple linear regression: more than one driver **Alex Tutor:** Back on CatChow's product team, we're not done with orders yet. One predictor was a nice start, but real orders depend on ad spend, holiday weeks, maybe price too. What if we used all of them together? **Sofia:** So we just add up their separate effects? **Alex Tutor:** Almost — multiple linear regression fits one equation with several predictors at once, each with its own coefficient. ![Three funnels, topped by a megaphone, a gift box and a price tag, pour into one large cat-food bowl while a cat watches.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/multiple-regression-illustration-2.webp) $$ \hat{y} = b_0 + b_1 x_1 + b_2 x_2 + \dots + b_k x_k $$ *Multiple regression: one intercept plus one slope per predictor, still fit by minimising squared residuals.* **Alex Tutor:** Each coefficient is the effect of its own predictor, holding all the other predictors constant — as if nothing else in the equation changed. **Sofia:** "Holding constant"? We can't actually freeze the other variables in real life. **Alex Tutor:** True — it's a mathematical statement about the fitted equation, not something you did during data collection. Still useful for comparing predictors fairly. ![A control panel with three sliders: two are locked with padlocks and a cat paw moves only the third, orange one.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/multiple-regression-illustration-3.webp) *Each coefficient answers: what changes if only this one input moves and the rest stay put?* > **Exercise:** In the equation orders = b0 + b1 x ad_spend + b2 x is_holiday_week, what does b1 represent? ## Reading a multiple regression equation **Alex Tutor:** Let's make this concrete with a CatChow-style multiple regression equation. **Alex Tutor:** $\text{orders} = 12 + 0.05 \times \text{ad\_spend} + 4 \times \text{is\_holiday\_week}$, where is_holiday_week is 1 in a holiday week and 0 otherwise. **Sofia:** So being a holiday week is worth a flat 4 extra orders, no matter the spend? **Alex Tutor:** Exactly, in this equation — a fixed bump on top of whatever ad spend predicts. > **Exercise:** Using orders = 12 + 0.05 x ad_spend + 4 x is_holiday_week, predict orders for a holiday week (is_holiday_week = 1) with ad_spend = 400. **Alex Tutor:** Nice. Now flip the holiday flag off. **Sofia:** Same spend formula, just drop the +4? **Alex Tutor:** Exactly that. **Sofia:** Let's check. > **Exercise:** Using orders = 12 + 0.05 x ad_spend + 4 x is_holiday_week, predict orders for a non-holiday week (is_holiday_week = 0) with ad_spend = 600. **Alex Tutor:** One more reading habit before we move on: people love to compare coefficients by size. Which comparison would you make? > **Exercise:** In orders = 12 + 0.05 × ad_spend + 4 × is_holiday_week, which comparison of the two predictors is **correct**? ## Why R² rises with every predictor **Alex Tutor:** Two coefficients down. But here's a trap: throwing in more and more predictors always makes R² look better on this data, even useless ones. **Sofia:** Wait, even random nonsense predictors help R²? **Alex Tutor:** By a tiny amount, yes — pure chance gives every extra column a little credit. That's exactly why adjusted R² and holdout testing exist. **Sofia:** So R² alone can be misleading with lots of predictors? > **Exercise:** Adding another predictor to a regression can only increase or leave unchanged its R² on the data it was fit on, even if the new predictor is pure random noise. **Alex Tutor:** Adjusted R² puts a price on every predictor you add. Try it on a five-predictor model fitted on twenty weeks. > **Exercise:** A model of weekly orders uses k = 5 predictors fitted on n = 20 weeks and has R² = 0.90. Compute its adjusted R² with $R^2_{adj} = 1 - (1 - R^2)\frac{n-1}{n-k-1}$. ## Multicollinearity: when predictors move together **Alex Tutor:** Good. There's a second, sneakier problem when predictors are correlated with each other, not just with the outcome. **Sofia:** Like ad spend and... what else moves together? **Alex Tutor:** Ad spend and discount depth, say — CatChow often deepens discounts during the same weeks it spends more on ads. > **Multicollinearity** > > Strong correlation between two or more predictors that makes their individual regression coefficients unstable and hard to interpret, even if the model's overall predictions stay fine. **Alex Tutor:** When two predictors move together that tightly, the model can't cleanly tell which one deserves the credit. Coefficients become unstable — add one data point and they can swing wildly. ![Two cats push one shopping cart side by side while a third cat with a magnifying glass tries to work out who pushed harder.](https://productovi.com/courses/statistics-for-pm-and-marketers/inline/multiple-regression-illustration-4.webp) *When two predictors always move together, the model can't tell which one deserves the credit.* > **Exercise:** Match each situation to the term that describes it. **Alex Tutor:** Good matches. Both problems have fixes. **Alex Tutor:** For multicollinearity: drop one correlated predictor, combine them, or fit a stepwise regression. For overfitting: penalise extra predictors with an information criterion, or test on holdout data. **Alex Tutor:** Here's what a holdout check looks like in numbers. Pick the model you'd ship. > **Exercise:** You fit two models on 80% of the weeks and test them on the remaining 20%. Model A (12 predictors): R² = 0.95 on training data, 0.55 on test data. Model B (3 predictors): 0.85 on training, 0.82 on test. Which should you use for next quarter's forecasts? **Sofia:** So more predictors isn't automatically a better model? **Alex Tutor:** Correct. More predictors is only better if each one earns its place. > **Exercise:** Which of these are true about multicollinearity? Select all that apply. ## Before you ship the formula **Alex Tutor:** Time to put this into words. **Sofia:** Here we go again. **Alex Tutor:** Quick one this time. **Sofia:** Fine, hit me. > **Exercise:** A teammate proposes adding both ad_spend and discount_depth to the model, noting CatChow usually raises both together during sales. What risk should you flag, and what would you suggest? Answer in 2-4 sentences. **Alex Tutor:** Good instinct — flag it, don't just ignore it. Now the full case. **Sofia:** Three predictors this time? **Alex Tutor:** Three predictors, and one multicollinearity risk hiding among them. Find it. **Alex Tutor:** Here's the scenario. > **Exercise:** Review the fitted model and write your recommendation. **Alex Tutor:** Solid work. You can now read a multiple regression equation, and you know its two classic traps. Next: what if the outcome itself is yes/no, not a number — logistic regression. - Previous lesson: [Simple linear regression](https://productovi.com/courses/statistics-for-pm-and-marketers/linear-regression.md) - Next lesson: [Logistic regression and classification](https://productovi.com/courses/statistics-for-pm-and-marketers/logistic-regression.md) --- Productovi Learning. Free, no sign-up needed to read.