What is a residual in a marketing mix model?

A residual is one miss: the gap between what your model predicted for a week and what actually happened. Every week has one. R-squared and MAPE compress all of them into a single fit score. Residual diagnostics keep the misses separate, so you can see where the model went wrong, not just how wrong it was overall. That is where good-looking models get caught.

Why doesn't a high R-squared mean the model is good?

Because R-squared and MAPE are aggregate summaries. They tell you how well the model fits overall. They say nothing about when the misses happen.

Two models can post the same average error. One misses a little all year. The other is spot-on most of the year and wrong every promotional week, always in the same direction. Same score, very different error structure, and the residuals are what make that difference visible.

A single fit number is also easy to put on a slide. Residual structure is harder to compress into one number, which is exactly why it is worth looking at on its own.

That matters because the misses pile up in the weeks you make your biggest calls: promotions, holidays, price changes, launches. A model that is quietly wrong there is handing you confident numbers in the moments budget actually moves.

A model can clear every fit number in the deck and still be wrong exactly where it counts.

What do residual patterns actually mean?

In a well-specified model, you do not want obvious structure left in the misses. When they line up with time, with promotions, or with the model's own predictions, the model is probably leaving something unexplained. The pattern shows you where to look. It does not tell you why, and it does not name a culprit.

Both patterns below can distort which channel gets credit, when the missing structure moves with your media.

Clustering. When the misses bunch up on promo weeks, the model has not captured what happens during a promotion. If those weeks also ran heavy media, the credit the model missed can attach to whatever moved alongside the promotion, media included, and a channel that rode a discount comes back looking better than it was.

Drift. When the misses slope up or down over time, the model has missed a trend, like rising baseline demand. Ad spend usually trends too, so the model can mistake that shared rise for media doing the work. Google's Meridian documentation flags this exact trap.

What does a hidden pattern cost you?

Say a brand runs a big Black Friday push. Sales spike, and so does spend on Meta, search, and email, all in the same few weeks. The model fits the year well and hands most of that November jump to Meta. The team scales Meta into January, when there is no promotion, and the lift does not follow.

Whether the residuals catch this depends on what the model did with the promotion. If it never fully accounted for the discount, the misses pile up on the Black Friday weeks, and the plot flags it. That is the good case: a warning before you act.

But the model can also fold the promo lift straight into Meta's number. Then the predictions match what happened, the residuals look clean, and Meta still gets credit it did not earn. The plot shows nothing, because nothing is left over to see. That is the case a residual check cannot catch, and it is why clean residuals are a floor, not a guarantee. Separating Meta from the promotion takes an experiment; the residual plot, however clean, cannot do that job.

Can a model look more certain than it is?

Yes, and this is the one to watch before you size a move. If the misses come in streaks, high for a run of weeks then low, the model's uncertainty can be understated, so the range around a number comes out tighter than the data supports. That is a known issue in time-series models.

The gap is not academic. A 2.4x return means one thing when the range around it is tight and something else when that range is wide enough to include a loss. If the model treated streaky errors as independent when they were not, the range it reported can be tighter than the data supports, and you sized a move on precision that was not really there.

You do not have to spot this yourself. You need to know the model checked for it, because a too-tight range is what makes a shaky number look safe to spend against.

Do clean residuals mean the model can predict?

No. A residual check looks at the weeks the model already learned from, so a clean result means it explained its own history. Predicting a week it has never seen is a separate test. A model can forecast total sales well and still get the individual channel numbers wrong.

Do clean residuals prove the channel split is right?

No, and this is the one that matters most before you reallocate. When two channels rise and fall together, the data holds less independent variation for telling their effects apart, however clean the residuals look. Pulling them apart takes stronger information from outside that history: an experiment, informative priors, or more independent movement in the spend. That is the collinearity problem, and it is why clean residuals are necessary but not enough to move budget between channels.

What does a good residual check look like?

Finding a pattern is where the work begins. A modeler treats it as a lead: form a guess about what is missing, add it to the model, refit, and see whether the pattern clears. Residuals bunch up on promos, so they add a promotion term and look again.

If the pattern goes away, that is evidence the change addressed what the plot flagged. It is not proof the guess was the whole story, but it is progress you can see. The next thing to check is what else moved: the channel estimates, the uncertainty around them, the baseline. If a number you were about to act on shifts, the problem mattered to your decision. If nothing decision-relevant moves, the fix still counted, it just did not change that particular call.

The point is that a real check leaves a trail: a pattern, a guess, a change, and a before-and-after you can inspect.

One more trap is worth knowing. A model can pass an aggregate fit check and still be misfitting individual weeks. Meridian's Bayesian posterior predictive check, for example, scores one discrepancy across the whole window, and Google's documentation warns that this kind of aggregate check can hide offsetting local misfits, one month high while another runs low. A passing global score does not replace looking at the fit over time.

How do you ask your vendor about this?

You do not have to read a plot. You have to ask to see one.

The question is: show me the residuals over time, tell me what pattern you found, what you changed because of it, and whether the channel numbers moved after. A vendor who did the work can show you all four. If instead you get a fit score, usually "the R-squared was high," that is not an answer about residuals at all.

That one request tells you more than any headline number on the page.

If they cannot produce it, that is your answer too. It does not prove the model is wrong. It proves nobody looked, and a number nobody checked is not one to move budget on. Ask for the residuals before the next reallocation, not after it goes sideways.

FAQ

What is a residual in marketing mix modeling?

A residual is the gap between what the model predicted for one week and what actually happened. R-squared and MAPE summarize fit across the whole dataset. Looking at the residuals directly shows you where the misses land, which the summary hides.

Can a model have a high R-squared and still be wrong?

Yes. A high R-squared means the model explained most of the variation in the past. It can still be wrong in specific windows, like every promotional week, and the summary will not show it. That is what residual patterns can surface.

What residual patterns should worry me?

Three. Misses that cluster on promos or holidays can mean the model missed something about those weeks. A slow drift up or down can mean it missed a trend. Misses that come in streaks can make the model look more certain than it is. Each is a reason to dig in, not a diagnosis on its own.

Do I need to read the residual plot myself?

No. You ask your vendor to show you the residuals over time, what pattern they found, what they changed, and whether the channel numbers moved. Producing that is the tell. A fit score offered instead is also a tell.

Do clean residuals prove which channel worked?

No. A model can look clean overall and still misjudge the channel split, especially when two channels always move together. Sorting that out takes an experiment or other outside information, not a better fit.

A budget call should never rest on one fit number. Residual checks are one of the questions in the nine-question set Stella runs on every model before treating its channel numbers as decision-ready. If you want to see what your current model is leaving in the misses, book a demo.