Much of what we read on MMM comes from MMM vendors, platforms and tool documentation. That is useful, but it has a point of view. This piece gives the decision in plain terms. It is one slice of the wider map in MMM vs MTA vs incrementality tests.
What does MMM answer that tests and attribution don't?
Marketing-mix modelling (MMM) estimates how marketing inputs contribute to business results, using aggregated data such as spend, sales, seasonality and price. It does not need user-level tracking. Underneath, it estimates a baseline: the sales you would expect without the marketing.
That makes it good at one kind of question: how should the budget be split across channels over time? It also looks at slower effects, such as adstock (carry-over of advertising after the spend stops) and saturation (diminishing returns).
It is not built for "did this ad work last week". For that, a holdout test gives a direct answer on one change. Attribution, meanwhile, divides credit by rule and does not show cause. MMM sits at a different altitude. It tells you about the mix. It does not replace the other two.
MMM can also point at marginal ROAS: the return on the next euro, not the average. That is the number a budget split really needs.
MMM answers budget-mix questions over time, not "did this ad work last week".
How much data does MMM need? (What the Robyn paper says)
A frequently cited source is the paper that introduces Robyn, Meta's open-source MMM. Two of its four authors work in Meta's Marketing Science team, so read it as the tool makers' view. It says that, customarily, the input data covers at least two years of weekly observations. It adds that for direct-to-consumer businesses and apps, shorter series of daily data can suffice1. Its rule of thumb is about ten observations per variable: a model with ten variables needs about 100 weekly observations, or roughly 180 days of daily data in the less-proven daily case1.
Treat that as a starting point, not a gate. It describes one tool's practice. Your own picture depends on three things:
- History. More weeks give the model more seasons to separate from marketing.
- Variation in spend. The same paper warns that two channels with largely constant spend over time are likely not suited to MMM1. The model needs periods where spend moved, ideally for reasons unrelated to demand.
- Number of channels. More channels with similar timing are harder to pull apart.
A Forrester analyst's post on Google's Meridian makes a related point. To build MMM yourself, he writes, you need strong in-house data engineering, data science and visualisation, plus trust between marketing and other departments2. Our view: if you cannot say which of your weeks had unusual promotions or stock-outs, your data is not ready.
What does it cost in analyst time and organisational trust?
Open-source tools remove the licence fee. They do not remove the work. Someone has to clean the data, choose the model structure, check the fit and explain the result to people who did not build it.
Trust is the bigger cost. Forrester's analyst notes that MMM results sometimes contradict campaign-level ROI and ROAS, because campaign-level measurement cannot account for what would have happened without the campaign2. That is expected, not a bug. Platforms count the conversions they can see and credit themselves. MMM estimates from total sales.
So you need a plan for the disagreement before the model lands. Who decides which number the budget follows? If the answer is "whoever shouts loudest", the model will be ignored.
The strongest answer to "why should we believe this?" is calibration. The Robyn paper calls calibration with randomised controlled trials the key way it mitigates bias1. Google's 2024 Meridian announcement lists calibration with incrementality experiments among its launch features, and says MMMs today are not perfect3. In practice, you may need incrementality tests anyway.
What are the known pitfalls of MMM?
In plain English, the pitfalls are:
- Coarse data. Weekly, national numbers hide regional and creative differences.
- Carry-over ignored. Without adstock, spend gets credit only in the week it happened.
- Omitted variables. If something drove sales and the model does not know about it (a price change, a stock-out, a viral moment), marketing gets the credit or the blame.
- Stability. INCRMNTAL argues MMM struggles with constant, frequent changes in activity4. A brand that changes channels every quarter gives it little to learn from.
- False precision. A model returns a tidy number. That does not make it right.
Open-source vs bought MMM: what changes?
This is the in-house MMM question. The pros and cons are mostly about who carries the work.
Open source (for example Robyn, Meridian and PyMC-Marketing). PyMC-Marketing is an Apache-2.0 licensed Bayesian MMM library with adstock, saturation and calibration from lift tests; its maker, PyMC Labs, sells consulting around it6. You get control and no licence fee. You also need analysts who can run and explain it. The vendor behind a tool also has an interest: a PPC Land commentary on Robyn asks who really benefits when an ad platform builds your MMM7. That is commentary, not proof of bias, but it is a fair question to ask of any tool.
Bought MMM (a vendor or agency builds and runs it). You get speed and fewer internal skills needed. You also get a model you may not be able to inspect, and a seller who benefits when you keep paying for it.
Our view: whichever route you pick, insist on seeing the assumptions, the data window and how it was calibrated. A model you cannot question is a number, not evidence.
What should you do before MMM makes sense?
Do the cheaper things first. They also make a later MMM better.
- Track MER (marketing efficiency ratio) properly. Our convention, not an industry standard: MER is total revenue divided by total marketing cost, including media, agency, creative and tools. Blended ROAS is total revenue divided by media spend alone. Pick one, define it in writing and keep it fixed.
- Record what changed and when. Promotions, price moves, stock-outs, new channels. This is the context an MMM will otherwise miss.
- Vary spend on purpose. A planned change in one channel or region teaches you something now and gives a future model something to learn from.
- Run targeted tests. A geo or audience holdout on your biggest or most doubtful channel is often the best next step. You will need them for calibration anyway.
- Write down the decision. Which budget choice would the model change? If you cannot name one, you do not need the model yet.
Example (illustrative): A brand spends the same amount on each of five channels every week for eighteen months. An MMM has almost nothing to separate them with. A brand that switched a channel off for four weeks in two regions has a cleaner signal, and that signal came from a cheap test.
Our view: MMM pays off for brands with long history, many channels, steady activity and analysts who will own it. Many mid-size brands are not there yet. That is fine.
Sources
- Packaging Up Media Mix Modeling: An Introduction to Robyn's Open-Source Approach (arXiv 2403.14674)Primary source
- Is Google's Meridian The Right Open-Source MMM Solution For You? (Forrester)Industry publication
- Meridian (blog Google, 7 mar 2024)Primary source
- The Rise and Fall of Media Mix Modeling (INCRMNTAL)Vendor
- Methodological pitfalls of traditional MMM (Sellforte)Vendor
- pymc-labs/pymc-marketing (GitHub README)Primary source
- Meta's Robyn: who really benefits when a platform builds your MMM (ppc.land)Blog or media
