Podcast page image

The Brandformance Podcast โ€ข Ep 64

The MMM jargon every marketing leader should know

With

In this episode of Brandformance's marketing-science segment, co-hosts Pranav Piyush (Paramark) and Sundar Swaminathan (ex-Uber, creator of ExperiMENTAL) give a plain-English tour of the jargon inside a marketing mix model โ€” the terms a CMO or VP of marketing should recognize without a stats degree. They demystify baseline vs. incremental (and why baselines always trend toward zero), control variables, ad stock and carryover, saturation and marginal ROAS, and the Bayesian-vs-frequentist debate that matters less than vendors imply. They explain why statistical significance and confidence intervals are about managing risk rather than objective truth, and untangle multicollinearity, endogeneity, counterfactuals and synthetic control. They close on how to tell whether a model is any good โ€” out-of-sample validation over vendor accuracy claims. The throughline: nearly every scary term is really a statistical stand-in for human behavior, and the honest answer is usually โ€œit depends.โ€

Episode Insights

Transcript

This is the marketing-science leg of Brandformance, co-hosted by Pranav Piyush and Sundar Swaminathan. Every two weeks they break down the concepts behind modern marketing measurement โ€” incrementality, marketing mix modeling, brand versus performance โ€” in plain, practical terms, and bust a few myths along the way.

Pranav Piyush is the co-founder and CEO of Paramark, where heโ€™s building modern marketing measurement โ€” incrementality testing and marketing mix modeling โ€” for growth teams. A long-time marketing leader, he hosts Brandformance.

Sundar Swaminathan is a marketing data science and experimentation advisor to consumer-tech scaleups and the creator of the ExperiMENTAL newsletter and podcast. He previously built and led brand data science at Uber โ€” measuring over a billion dollars of brand spend โ€” and earlier worked on the debt desk at the US Treasury.

The gist

  • A plain-English tour of the jargon inside a marketing mix model โ€” the terms a CMO or VP of marketing should recognize without needing a stats degree.

  • Almost every term is a statistical representation of human behavior โ€” and almost none has a single โ€œrightโ€ definition. The honest answer is usually โ€œit depends.โ€

  • Baselines arenโ€™t fixed โ€œorganicโ€ demand; theyโ€™re the residue of past marketing, and they always trend toward zero.

  • Statistical significance and confidence intervals arenโ€™t objective truth โ€” theyโ€™re ways to manage risk.

  • Models estimate and build confidence; they donโ€™t predict the future. Judge them on out-of-sample performance, not vendor accuracy claims.

Baseline vs. incremental โ€” and why baselines trend to zero

The most fundamental โ€” and most misunderstood โ€” term. Baseline is what would have happened without marketing; incremental is the additional business marketing drove. But that only holds in the short term (think three to six months), because todayโ€™s baseline was itself built by years of past marketing. Sundarโ€™s Nike example: cut brand for years and the whole baseline shifts down.

Pranavโ€™s sharper framing: there is no fixed โ€œorganicโ€ demand sitting out in the world. Stop everything and the baseline eventually goes to zero โ€” even a box of cereal loses its shelf placement and visibility once it stops selling. A CFOโ€™s โ€œwhat would we make if we cut marketing to zero?โ€ is fair for a moment, but youโ€™re just borrowing from future growth. Statisticians define baseline as whatever the model couldnโ€™t explain (the residual) โ€” technically true, but itโ€™s standing in for the same idea. Every definition is partly right, partly wrong.

Control variables โ€” what marketing doesnโ€™t control

Control variables are the things that moved your business but werenโ€™t your marketing decisions โ€” the model needs them to isolate marketingโ€™s true effect. Ironically, theyโ€™re the things marketing canโ€™t control: seasonality, price changes, product rollouts, even a one-day Cloudflare outage that quietly drops a month 3โ€“4%.

The list is business-specific: FX rates for fintech, crypto sentiment for crypto, weather for ice cream. The discipline is picking the real needle-movers rather than throwing the whole kitchen sink at the model, which just invites misleading correlations.

Carryover and ad stock โ€” marketingโ€™s memory

These sound like pure statistics but are really representations of psychology โ€” how long people remember your brand after you stop advertising. Turn off Nike today and people still name it tomorrow; ad stock/carryover is the modelโ€™s attempt to capture that delayed, decaying effect. Higher mental availability generally means a longer carryover, because competitors take longer to absorb the space you vacate. The name is the metaphor: like grocery stock, the inventory in a buyerโ€™s mind draws down gradually rather than vanishing at once โ€” along a curve.

A practical warning from Sundar: people forget to test for ad stock. Run a three-month cut and measure only those three months and youโ€™ve ignored the residual effect of the marketing that came before โ€” and if your conversion cycle is six to eight weeks, youโ€™ve effectively mismeasured the window. Donโ€™t pull a geo test early just because the first weeks still look fine; thatโ€™s leftover ad stock, not the final answer.

Saturation, diminishing returns, and marginal ROAS

Saturation says each additional dollar does less than the last โ€” because your earliest buyers are the most problem- and solution-aware, and every later cohort is harder and costlier to convert. Even Meta, twenty years in, still faces it. And the curveโ€™s shape differs by business: AI may be bending it in a way we havenโ€™t seen historically, while retailโ€™s is far steeper.

Marginal ROAS is the return measured at the current point on that curve, not averaged across the whole thing โ€” so a dollar spent where the curve is still steep looks very different from one where itโ€™s flattening. Sundar adds the other half: saturation is really the cost curve, while ROAS is the intersection of the cost curve and a value (LTV) curve, since later cohorts are also worth less. Spend up to where what you get back equals what you put in โ€” and remember that at very high spend, auction competition itself pushes costs up.

Bayesian, significance, and intervals โ€” risk, not truth

On frequentist vs. Bayesian, both hosts cheerfully admit the limits of their stats depth โ€” and argue that for a marketing leader it mostly doesnโ€™t matter. Bayesian starts with a prediction (a โ€œpriorโ€) and updates as data arrives; frequentist leans on assumptions about distributions. Modern MMM and incrementality vendors have largely standardized on Bayesian, partly because it lets you feed incrementality-test results in as priors. Useful to know, but not a reason to stall.

Statistical significance gets mistaken for objectivity because it says โ€œstatistical.โ€ But 95% and 99% are essentially arbitrary historical conventions; significance is really how much risk youโ€™re willing to accept. At Uber, moving zero-cost email tests from 95% to 80% two-tailed let them decide faster and test more, and no one cared. And the pedantic-but-real distinction: a โ€œconfidence intervalโ€ (frequentist) doesnโ€™t actually mean a 95% chance the true value sits in the range โ€” thatโ€™s a โ€œcredible intervalโ€ (Bayesian). The terms matter less than knowing they describe risk, not ground truth.

Multicollinearity and endogeneity โ€” when the model canโ€™t tell channels apart

Break the scary word down: multi-collinear = multiple things moving together. If a team peanut-butters budget โ€” bumping Meta and Google by the same percentage every month โ€” the two channels move in lockstep and the model canโ€™t separate their effects. No variability, no attribution.

Endogeneity is the cause-and-effect loop. Branded search is the poster child: you donโ€™t control the auction, demand does โ€” so demand drives spend, not the other way around. Run a lot of TV and branded search suddenly looks great, because TV created the demand the model then credits to search. Both terms describe the same underlying problem: relationships in the model that are genuinely hard to untangle into โ€œwhat moves what.โ€

Counterfactual and synthetic control

A counterfactual is simply the reverse of your decision โ€” what would have happened if you hadnโ€™t (or had) done the thing. Itโ€™s the comparison you need to measure impact. In a clean A/B test, an email test, or a geo test, you can observe the counterfactual directly. In many real situations, you canโ€™t.

Thatโ€™s where synthetic control comes in: you estimate the counterfactual by stitching together a โ€œFrankensteinโ€ comparison from other markets. Testing in New York with no clean twin? Build a synthetic New York from the useful pieces of San Francisco, Philadelphia and others so the combined line tracks New York before the test โ€” then attribute the gap that opens up during it. Itโ€™s not causal proof; itโ€™s a made-up but disciplined stand-in for the control you couldnโ€™t run.

Is the model any good?

MAPE, RMSE, in-sample vs. out-of-sample, backtesting, overfitting โ€” all of it is asking one question: can the model predict data it wasnโ€™t given? Models are dangerously good at forcing a pattern onto the data theyโ€™ve seen (in-sample); the real test is out-of-sample, predicting a past period that was hidden from it. Do that well and you have more confidence it can handle the genuinely unknown future.

For a CMO, the job isnโ€™t to memorize the metrics โ€” itโ€™s to make sure the data-science team actually did the rigorous validation, especially when thereโ€™s pressure to ship. And beware false precision: no model saw the Gulf conflict or the gas-price spike coming, so treat these as estimation and confidence-increasing machines, not prediction machines. Chasing โ€œ94% is better than 92%โ€ is exactly the kind of vendor jargon to be skeptical of.

Quote snacks

  • โ€œMarketing and baselines in general are always trending towards zero.โ€ โ€” Sundar

  • โ€œThereโ€™s no organic โ€” everything is made up, everything is artificial.โ€ โ€” Pranav

  • โ€œA lot of it does depend. Thereโ€™s no clear-cut definition for anything โ€” thatโ€™s why MMMs are challenging.โ€ โ€” Sundar

  • โ€œStatistical significance is how much risk youโ€™re willing to accept in an uncertain decision.โ€ โ€” Sundar

  • โ€œTreat the model a bit like a black box, and let someone who can unpack it do more of that thinking for you.โ€ โ€” Sundar

  • โ€œTreat these as estimation machines and confidence-increasing machines โ€” not prediction machines.โ€ โ€” Pranav

Why it matters

MMM is becoming a default layer of the modern measurement stack, and the jargon is a big part of why marketing leaders keep it at armโ€™s length. Demystifying the vocabulary โ€” and showing that most of it reduces to human behavior and risk tolerance โ€” is what lets a CMO ask the right questions instead of deferring entirely or bluffing along.

It also reinforces the seriesโ€™ core stance: models are tools to be interrogated, not oracles to be obeyed. Know what baseline, ad stock, saturation and significance actually represent, insist your team validates out-of-sample, and you can treat an MMM as a confidence-building instrument rather than a black box you either worship or ignore.

Practical next steps

Stop treating baseline as fixed โ€œorganicโ€ demand. Read it as the residue of past marketing, and expect it to decay if you stop investing.

Choose control variables deliberately. Add the real needle-movers for your business โ€” not the whole kitchen sink โ€” so the model doesnโ€™t chase noise.

Respect ad stock in your tests. Add your conversion-cycle window to the end of a test, and donโ€™t pull a geo test early on residual effects.

Set significance by risk, not habit. Match your confidence threshold to the stakes of the decision instead of defaulting to 95%.

Watch for lockstep spend. Vary channel budgets so the model can actually separate them, and flag endogenous channels like branded search.

Insist on out-of-sample validation. Ask your team to prove the model predicts data it wasnโ€™t trained on, and stay skeptical of vendor accuracy claims.

The sharpest marketing conversations, straight to your inbox

By providing your contact info, you agree to receive communications from Paramark. You can opt-out at any time. For details, refer to our Privacy Policy