Insights
What to do after you've run the geo-test
A geo-test readout is where the real work starts: how to turn results into budget decisions, communicate them clearly, and handle the politics of cutting spend.
Sundar Swaminathan
Marketing Science Advisor
What to do after you've run the geo-test
There’s a lot of content on how to design experiments. Far less about what to do when the results come back, especially when they’re inconvenient.
This article is about what happens next: turning a geo-test result into a budget recommendation, explaining that recommendation to stakeholders, and handling the politics when it means cutting spend on a channel someone owns.
What a geo test tells you
A geo test works much like an A/B test, except the units assigned to test and control are geographic areas rather than individual people. For the purpose of this article, I’m going to assume the test was set up properly, including pre-validation and a power analysis. That’s not always a safe assumption, but test design is a separate conversation.
The readout should give you an estimated effect on a metric you chose ahead of time (often transactions or revenue) alongside a confidence interval. The estimate gives you the effect size; the interval shows how much uncertainty there is around that estimate.
For a test measuring the difference between test and control, the interval might:
Fall entirely below zero, suggesting the test performed worse on the measured outcome.
Cross zero, meaning the data don’t clearly establish which performed better.
Fall entirely above zero, suggesting the test performed better on the measured outcome.
Let’s use an example: say you cut spend by 25% in the test geos to see whether revenue holds up. The estimated revenue difference comes back at zero, with a confidence interval spanning a 5% decrease to a 5% increase.
That’s promising, but it doesn’t prove the cut had no effect. A revenue decline of up to 5% is still consistent with the result. Whether you keep the cut depends on how much downside the business is willing to accept in exchange for the savings.
So how do you turn that result into a decision and get the rest of the business on board?
I think of it in three steps:
Translating findings into a budget decision
Communicating the recommendation clearly
Acting on the recommendation
Translating findings into budget decisions
A geo test result isn’t a yes or no. It’s telling you something about the marginal return of a specific change in spend. Sometimes you’re testing the marginal ROI of an entire channel (a channel incrementality test). Sometimes you’re testing the marginal ROI of a specific dollar amount up or down, usually called a pulse test.
Either way, the result is telling you what those dollars are worth.
And again, it’s a range, not a single number. Your estimate might imply a 40% ROAS, for example, but the confidence interval could tell you the true effect is plausibly somewhere between 10% and 80%. That’s useful because it naturally gives you a set of scenarios when you’re talking to Finance: worst case, base case, and best case.
Worst case = lowest end of the range
Base case = point estimate
Best case = highest end of the range
For each scenario, show:
The estimated dollar impact.
Whether it clears the threshold for a budget change.
What that change would mean for the business overall.
That last step keeps the discussion from getting stuck at the channel level. A cut that looks attractive in a marketing dashboard may look different once you account for its possible effect on total revenue and contribution margin. The size of the paid marketing program matters, too: changing spend in a small part of the business is a different decision from changing spend in one of its main growth engines.
Take the 25% cut example above. The best estimate says revenue stays flat, but the range includes a drop of up to 5%. What would that drop cost you, and would you still save money after accounting for it? Show what the cut could mean for CAC and payback, too.
Once you’ve translated the result into business impact, you still have to decide what to do with it. And this is the part where teams usually make their biggest mistake: they wait until the result comes back to decide what the result means. If the result says cut, scale, or reallocate, you should already have a threshold for each outcome. Otherwise, every readout turns into a negotiation.
When the confidence interval is too wide
Sometimes the point estimate is positive, but the confidence interval is so wide it’s hard to act on with any real conviction. This often happens with underpowered tests.
There isn’t a clean step-by-step answer here. You have to decide how much uncertainty you’re willing to accept.
You can say, “We’re going to retest because the confidence interval is too wide to make a confident decision.” Or you can proceed anyway, while acknowledging the uncertainty: “The impact could be smaller than the point estimate suggests.”
Neither approach is right or wrong. It depends on the size of the potential impact, the cost of acting on the result, and how much uncertainty the business is willing to accept.
The important thing is to make that uncertainty explicit. This is a statistical analysis, and part of your job is translating what the statistic actually tells you into something the people making the decision can use.
Communicating the recommendation clearly
Lead with what you recommend doing and why it matters to the business. If you start with the methodology, stakeholders have to work out the decision for themselves.
The test is evidence for the recommendation, not a substitute for judgment. Explain how the result compares with the threshold you agreed on, what it could mean for the business, and where uncertainty remains. That gives people a basis for the decision without making the estimate sound more definitive than it is.
Then say what happens to the budget. If you’re recommending a cut, will that money return to the general marketing pool, fund another channel, or stay unspent? If that hasn’t been decided yet, say who will decide and when. Stakeholders need to understand the proposed action, not just the test result.
Handling the politics when the data says to cut a channel
Experimentation is a human endeavor. When things are going well, everyone’s happy to be along for the ride. But when the data goes against expectations, you have to think carefully about how you communicate it.
But why is communicating a bad result so hard?
First, the result can make people question past decisions.
Unlike Product, where you generally A/B test before you launch, Marketing often launches first and tests later. I’m talking specifically about performance marketing and larger campaigns. You’ve tested pieces of the strategy, but you haven’t necessarily tested the actual budget level.
So when a result comes back that challenges what everyone assumed, people naturally start asking: How long have we been doing this? How much have we been overspending, and for how long?
The person who owns the channel can suddenly feel like they’re being told they’ve been wasting money. That’s not necessarily what the test says, especially in organizations with a good testing culture, but it’s an understandable human reaction.
Second, depending on how you present the result and what exactly you tested, it can feel like the entire channel doesn’t work, which means the strategy doesn’t work, which means the person running it isn’t good at marketing. That’s a much bigger claim than the test can support, but it’s an easy leap for people to make.
Give the channel owner (and everyone else) a more accurate way to talk about it: “We’re not saying the whole channel is ineffective. We’re saying the spend we tested didn’t meet the bar we agreed on.”
This is exactly why pre-alignment matters. It doesn’t remove the emotion, but it gives everyone a decision rule to come back to. If you’ve got a marketer who’s willing to say up front, “Yes, I don’t know if my budget is perfectly optimized,” you walk into the test with decision criteria already agreed on. Without that alignment, the conversation gets defensive fast.
Acting on the recommendation
The final step is the simplest to describe and one of the most important: actually do what you said you would do.
People have long memories. If you tell the organization you’re going to cut a channel, reallocate the budget, or monitor a change, they expect you to follow through.
Once the change has been made, close the loop with everyone involved. Over the next few months, monitor performance and show what happened. If contribution margin improves or payback gets better, share that too. It shows that you didn’t just run a test and move on. You made a decision, followed through, and measured the outcome.
This is where the real cost of ignoring a geo-test shows up.
If you cave to internal pressure and ignore the result, you’ve taught the organization that your measurement is negotiable. The next time you run an experiment, people know they can challenge the result until they get the answer they want.
That costs you far more than one channel owner’s discomfort.
Stay experiMENTAL, Sundar
While you're here
Find your edge
Book a demo now and you'll get a live, expert-run walk through of how Paramark can help you with incrementality measurement and experimentation.
