Measurement science

Next measurement method after weak budget evidence

Published June 3, 2026. Updated June 3, 2026. Status: evergreen explainer.

A budget worksheet is useful when it shows that the evidence is not strong enough for the action being requested. The next step is not to ask for a louder dashboard. It is to choose the measurement method that answers the missing question.

Use this guide after the budget decision evidence ladder or budget decision meeting worksheet flags a weak evidence lane. The choice usually comes down to four designs: MMM calibration, geo lift, randomized holdouts, or a brand study.

First name the gap

The method should match the missing counterfactual. If the report cannot show what would have happened anyway, ask which comparison would change the budget decision.

Weak evidence lane	Best next method	Use when the decision is	Do not use it to prove
Planning assumptions across channels, seasons, or spend levels are weak.	MMM calibration.	How much budget should move across a portfolio, with ranges and assumptions visible.	That a single campaign caused a specific conversion path.
The action is market, region, store, or territory based.	Geo lift or matched-market test.	Whether a market-level change produced incremental outcomes beyond comparable markets.	That every market, season, or spend level will behave the same way.
The action needs person-level or household-level conversion lift.	Randomized holdout.	Whether exposed eligible users changed behavior against a protected control group.	That the result automatically generalizes beyond the assigned population and window.
The question is awareness, recall, consideration, or favorability.	Brand study.	Whether surveyed perception moved inside a defined population.	That perception movement caused sales, profit, or pipeline lift.

Decision chooser

MMM calibration

Use for portfolio planning

Choose this when the budget question spans channels or quarters and a model needs credible anchors. The evidence request should ask which experiments, geo tests, holdouts, or external priors calibrate the model, how sensitive the ranking is to those anchors, and where uncertainty changes the decision.

Geo lift

Use for market-level actions

Choose this when treatment can vary by market, store, region, or territory. The evidence request should ask for pre-period balance, matched controls, local shocks, spillover checks, outcome maturity, and the rule that decides whether the test changes the budget.

Randomized holdout

Use for controlled exposure questions

Choose this when eligible users, households, accounts, or devices can be assigned before exposure and the control can be protected. The evidence request should ask for assignment logs, suppression rules, leakage checks, base rates, minimum detectable effect, and uncertainty.

Brand study

Use for perception outcomes

Choose this when the decision is about message memory, awareness, consideration, or favorability. The evidence request should ask for sample source, exposed-control balance, question wording, field dates, weighting, respondent quality, and the exact budget decision the survey can inform.

If the budget decision cannot wait

Sometimes the decision window arrives before the better method can be run. In that case, the action should stay below the evidence rung and the next evidence request should be explicit.

Current evidence	Safer interim action	Next evidence request
Clean delivery and exposure, weak outcome evidence.	Repair setup, preserve clean inventory, or renew only with narrow operational language.	Outcome-quality review, baseline comparison, or randomized holdout if lift language is needed.
Observed response is strong, but no comparison is visible.	Renew cautiously or keep the same scope while asking for a stronger counterfactual.	Protected holdout, geo test, or baseline comparison that shows what would have happened anyway.
Directional comparison favors the campaign, but uncertainty is material.	Make a bounded budget-direction call only if the low end still clears the business threshold.	Uncertainty review, minimum detectable effect plan, and a preplanned test for the next material change.
Model or survey evidence supports the story, but method details are thin.	Use the result as directional context, not final budget proof.	MMM calibration detail or brand-study method disclosure before stronger language is used.

Method-specific evidence requests

Method	Ask for	Reject as insufficient
MMM calibration	Calibration anchors, priors, controls, response curves, uncertainty ranges, holdout or geo evidence, and sensitivity to exclusions.	A high fit score or channel ranking without calibration and uncertainty.
Geo lift	Market eligibility, matching variables, pre-period levels and trends, local events, spillover risk, outcome window, and readout threshold.	A post-period market gap with no pre-period balance or local-shock check.
Randomized holdout	Assignment unit, eligible universe, suppression proof, treatment compliance, control leakage checks, base rate, power, outcome maturity, and interval.	A platform lift label without assignment, leakage, base-rate, and uncertainty detail.
Brand study	Sample source, exposed and control balance, survey timing, question wording, weighting, respondent quality, outcome meaning, and uncertainty.	A single lift percentage that does not show who answered or what changed.

How to write the next action

Good decision language names the method and keeps the action inside its boundary. A useful sentence sounds like this:

Decision language

The current readout supports a cautious renewal and a stronger next test. Because the budget question is market-level, the next evidence request is a matched-market geo lift plan with pre-period balance, local-shock checks, outcome maturity rules, and a threshold for whether the result changes spend.

If the method is MMM calibration, replace the market language with calibration anchors, priors, sensitivity, and planning ranges. If the method is a randomized holdout, name assignment, suppression, leakage, base rate, and uncertainty. If the method is a brand study, keep the claim to surveyed perception and do not use it as sales lift.

Pair with

Start with the measurement method selector when the method is still unclear. Use the budget decision evidence ladder to choose the evidence rung, the budget decision meeting worksheet to record the action, the MMM calibration evidence checklist for model anchors, the geo lift test design checklist for market tests, the randomized lift test readout checklist for holdout readouts, and the brand lift study readout checklist for survey evidence.