Marketing mix modeling
The MMM causal validity checklist
Marketing mix modeling is useful because it can inform budget allocation when user-level tracking is incomplete. It is dangerous when teams treat a model that predicts well as a model that assigns causal credit correctly.
Google's Meridian documentation states that the primary goal of MMM is estimating causal marketing effects, while also warning that directly validating causal inference is difficult and usually requires well-designed experiments with the same estimand. That is the right starting point: prediction is not enough.
Checklist
1. Estimand
Does the model define the outcome it is estimating: incremental revenue, conversions, store visits, brand demand, or another response? If the decision is budget allocation, the estimand must match that decision.
2. Confounders
Does the control set include variables that affect both media execution and business outcome: seasonality, pricing, promotions, distribution, macro shocks, competitor activity, and baseline demand?
3. Over-control risk
Controls are not trophies. Adding variables that are consequences of marketing, rather than confounders, can bias estimates and hide real effects.
4. Priors and constraints
Does the model disclose priors, channel constraints, response curve assumptions, and adstock assumptions? Hidden priors can quietly decide the answer.
5. Calibration
Are credible lift experiments, geo tests, or holdout studies used to calibrate the MMM? If so, do the experiments estimate the same effect the MMM is asked to estimate?
6. Uncertainty
Does reporting show uncertainty intervals, not just point estimates and rank ordering? A channel can appear best while remaining statistically indistinguishable from a peer.
7. Decision sensitivity
Would the recommended budget move change if priors, controls, or time windows changed within reasonable bounds? If the recommendation is fragile, say so.
Red flags
- The vendor leads with MAPE or R-squared and never discusses causal identification.
- The model ranks every channel with crisp ROAS numbers but no uncertainty.
- Seasonal promotions, pricing, or distribution changes are missing from controls.
- Platform-reported conversions are used as truth without incrementality checks.
- One lift test is used to calibrate a different audience, channel, outcome, or time window.
Primary references
Google Meridian documentation on model fit and causal inference explains why MMM decisions should be evaluated through causal reasoning, not only predictive accuracy. Google's Meridian launch note describes Meridian as an open-source MMM for modern marketing measurement.