Measurement science

Advertising measurement method selector

Organization byline: Measurement Press editorial desk. This page does not list an individual author or reviewer. Read the editorial method and corrections policy.

About 13 min read

On this page

Most measurement mistakes begin before the analysis starts. A team asks one question, buys a method built for another question, and then treats the output as if it settles the decision.

A common version looks harmless: the sales report says revenue rose after launch, the attribution report gives one channel most of the credit, the brand study shows a small recall gain, and the budget owner asks what to do next. Each signal may be useful. None of them can answer every question.

This selector is a practical way to slow that down. Start with the decision, name the counterfactual, choose the method that can answer it, and keep every result inside the boundary of what the design can support. In plain language, the counterfactual is the comparison that says what probably would have happened if the campaign had not run.

Editorial measurement desk showing several report cards pointing toward one central budget decision folder. — Method choice starts when several reports point at one decision. The useful question is not which report looks strongest, but which report has the right comparison for the action being considered.

Budget follow-up: if the budget decision worksheet shows the current evidence is below the needed rung, use the next-method explainer to choose between MMM calibration, geo lift, randomized holdouts, and brand studies.

Start with the decision

Do not begin with "we need MMM" or "we need a lift test." Begin with the sentence the decision owner will use after the readout. A method that is strong for one sentence can be weak or misleading for another.

Before a vendor brief, analytics request, or internal readout, write a short selection ticket. It keeps the method tied to the business choice instead of letting the available dashboard define the question.

Selection ticket field	Write this before choosing the method	Why it prevents overclaiming
Decision owner	The person or team that will change a budget, bid, creative plan, launch plan, or renewal language.	Prevents measurement work from becoming an interesting report with no practical action.
Action under review	Scale, hold, pause, renew, shift mix, change creative, fix tracking, or run a stronger test.	Different actions need different confidence. A diagnostic may be enough for repair, but not for scale.
Counterfactual	The comparison that represents what would probably have happened without the campaign or change.	Forces the team to separate after-launch movement from incremental impact.
Primary outcome	One outcome tied to the decision, such as incremental conversions, qualified leads, revenue, recall, or reach quality.	Stops the readout from switching to whichever metric looks best after results are visible.
Minimum useful effect	The smallest change that would justify the action after cost, risk, timing, and uncertainty are considered.	Turns a "positive" result into a decision threshold instead of a celebration of any upward movement.
Decision deadline	The date when the evidence must be ready for planning, renewal, launch, or closeout.	Clarifies whether a quick diagnostic, a live holdout, or a longer modeling plan is realistic.

Infographic model connecting a decision card to a counterfactual comparison, a method toolkit, and a separate limit boundary. — The selector has four parts: define the decision, choose the comparison, use the method that can read that comparison, and keep the claim inside the method's limit.

Decision question	Usually stronger method	What it still cannot prove alone
Should next quarter's budget move between channels?	MMM calibrated with credible experiments where available.	That every channel estimate is causally identified or stable across future conditions.
Did this campaign create incremental conversions?	Randomized conversion lift test with protected holdout and a fixed outcome window.	That the same lift will hold for a different audience, creative, season, or bid strategy.
Did a market-level launch or retail push work?	Geo, store, or matched-market test with balanced pre-period trends.	That one strong region represents the national average.
Did the campaign change awareness or consideration?	Brand study with comparable exposed and control respondents.	That survey movement caused profitable sales movement.
Which impressions were more likely to be noticed?	Attention measurement used as an exposure-quality diagnostic.	That attention is the same thing as incremental business value.
Which touchpoints appeared in observed paths?	Attribution or path reporting, labeled as descriptive.	That the last or weighted touch caused the conversion.

The method matrix

MMM

Best for budget planning when spend, outcomes, prices, seasonality, distribution, and other business drivers can be modeled over time. It needs transparent controls, priors, response curves, uncertainty, and calibration. Predictive fit is not enough for causal confidence.

Randomized lift test

Best when assignment can be controlled and treatment and holdout groups can be kept meaningfully separate. The readout should name the estimand, exposure compliance, primary outcome, confidence or credible interval, and generalization limits.

Geo or store test

Best when treatment is applied by location rather than person. The core check is whether treatment and control areas had similar pre-period levels and trends before the campaign changed anything.

Brand study

Best for directional perception outcomes such as awareness, recall, favorability, and consideration. It becomes weaker when response bias, small samples, proxy outcomes, and sales claims are hidden behind a single lift number.

Attention measurement

Best for diagnosing exposure quality across placements, formats, creative, and environments. It is useful when paired with a decision rule, but it should not be treated as a universal currency for business outcomes.

Attribution reporting

Best for describing observed paths, handoffs, and operational tracking gaps. It can guide QA and tactics, but it should not be used as the main proof that media caused demand.

Source check for method choice

The boundaries in this matrix are consistent with the method owners' own documentation. Google describes lift studies as controlled treatment-and-control comparisons and distinguishes user-based from geo-based conversion lift. Its Meridian documentation describes MMM as a framework for contribution, response curves, and budget planning, with priors, control variables, lag, saturation, and model-health review affecting the inference. Google Brand Lift documentation describes perception outcomes such as recall, awareness, and consideration by comparing people who saw ads with eligible people who did not. Its attribution guidance explains how models distribute credit across conversion paths; that is different from estimating incremental impact. For attention, the IAB/MRC framework describes transparency, validation, and business-alignment requirements and says attention is a complementary signal, not a standalone effectiveness measure.

Source-to-decision handoff

These sources establish method boundaries; they do not validate a particular campaign result. Use the source description to choose the next check, then make the claim no broader than the design and data allow.

Editorial source verification: The linked documentation was re-read for this source trail. Google’s lift guidance supports the treatment/control distinction and user-versus-geo choice; Meridian’s overview supports recording the KPI, media inputs, controls, priors, and posterior outputs; Brand Lift’s documentation supports treating recall, awareness, consideration, and related survey measures as perception outcomes; attribution documentation supports describing credit allocation under a selected model; and IAB/MRC supports using attention alongside delivery and outcome evidence. None of these sources validates the simulated figures, establishes a universal method winner, or turns a platform report into proof of a real campaign result.

Method	What the source supports	Operational check before readout	Claim ceiling
Conversion or geo lift	Google describes treatment/control comparisons and user- or geography-based conversion lift.	Confirm assignment, exposure separation, one primary outcome, and a fixed outcome window.	Estimated difference for the defined population, treatment, and window; not a universal future lift.
MMM	Meridian is built around contribution, response curves, and budget planning with controls and priors.	Record KPI, media inputs, control variables, priors, model health, and credible intervals.	Planning inference within the model, data, and assumption range; not a substitute for every experiment.
Brand lift	Google frames Brand Lift around perception measures such as recall, awareness, and consideration.	Check exposed and comparison response counts, survey metric, eligibility, and response comparability.	Perception movement in the measured study; not proof of profitable sales impact.
Attribution	Google describes attribution as distributing conversion credit across interactions and informing optimization.	Record the model, conversion action, date range, eligible channels, and tracking QA status.	Observed or modeled credit under that setup; not causal incrementality by itself.
Attention	IAB/MRC describes definitions, methodological approaches, disclosure, validation, and complementary use.	Name the approach, measurement inputs, validation status, and business KPI it is meant to inform.	Exposure-quality context; not a standalone effectiveness or outcome measure.

A method lane is useful because it says what kind of question is being answered. Budget planning, campaign incrementality, market launches, brand memory, exposure quality, and observed paths should not be collapsed into one score.

Minimum disclosure checklist

Every serious report should state	Why it matters
The decision the result is meant to inform.	A method is only useful if it changes a real planning, buying, creative, or product choice.
The counterfactual being estimated.	Readers need to know what the campaign is being compared against.
The unit of analysis and unit of assignment.	User, household, market, store, and time-window designs have different leakage risks.
The primary outcome and readout rule.	Prevents a positive-looking secondary metric from replacing the planned decision threshold.
Important exclusions, model assumptions, and uncertainty.	Lets readers distinguish a robust result from a fragile point estimate.
Where the result should not be generalized.	Keeps a narrow but useful finding from becoming an inflated planning claim.

How methods work together

The strongest measurement programs do not force every question through one tool. Attribution can reveal broken tagging. Lift tests can calibrate causal expectations. MMM can turn multiple signals into planning ranges. Brand studies can explain whether a campaign changed memory or perception before sales data is mature.

The discipline is to keep each signal in its lane. A brand lift study should not become proof of profitable sales lift. A path report should not become incrementality proof. An MMM should not hide the experiment evidence that would make its causal claims stronger or weaker.

Worked example: before-after lift is not enough

Suppose a campaign launches in June and qualified leads rise by 3% compared with May. That is an observed change, not yet a causal result. A 3% increase might be useful, but it could also come from seasonality, a pricing change, a sales-team follow-up push, a landing-page fix, or customers who were already going to convert.

The method choice depends on the action. If the team only needs to debug tracking or qualify the next creative test, a descriptive readout may be enough. If the team wants to scale the campaign budget, the comparison has to get stronger.

Observed situation	Do not say yet	Better next step	Decision language after the step
Leads rose 3% after launch, but no control group exists.	"The campaign drove a 3% lift."	Check baseline stability, marketing calendar, sales follow-up, lead quality, and tracking changes.	"Leads increased after launch, but causality is not established."
The comparison period is seasonally weak or operationally different.	"The new media mix is outperforming the old mix."	Use a matched period, geo comparison, store comparison, or model with explicit controls.	"The result is directional until comparison quality improves."
The campaign can protect a holdout or randomize eligibility.	"Attribution proves the winner."	Run a randomized lift test with one primary outcome and a fixed readout window.	"The test estimates incremental impact for this audience, window, and treatment."
The action is a broad budget shift across channels.	"One campaign test settles next quarter's mix."	Use experiments to calibrate MMM or a planning model that includes non-media drivers.	"Budget movement is supported within the model range and calibration limits."

Simulated method-choice gate

This is a Simulated Worked Example, not a Measurement Press campaign result. Assume a team must decide whether to renew a campaign for a defined audience. The eligible population is 1,500 units: 1,200 treatment units and 300 holdout units. The primary outcome is qualified leads within a fixed 30-day window.

Gate	Simulated evidence	Score	Decision effect
Decision	Renew only if the campaign produces at least 15 incremental qualified leads at the approved cost threshold.	2/2	The method is selected for a real action, not a generic performance question.
Counterfactual	1,200 treatment + 300 holdout = 1,500 eligible units; holdout share = 300 / 1,500 = 20.0%.	2/2	A protected comparison exists, subject to assignment and leakage checks.
Outcome	96 qualified leads / 1,200 = 8.0% treatment; 18 / 300 = 6.0% holdout; difference = 2.0 percentage points.	2/2	The outcome and denominator match the renewal decision.
Feasibility	Assignment fields exist, but power, conversion lag, and minimum detectable effect have not been reviewed.	1/2	Use the result cautiously until precision and lag checks are complete.
Generalization	The simulated comparison covers one audience and one campaign window, not the full channel mix or next quarter.	1/2	Do not use this single result to settle a broad budget allocation.

Simulated arithmetic: 8.0% - 6.0% = 2.0 percentage points; 1,200 x 0.02 = 24 illustrative incremental leads.

The gate score is 2 + 2 + 2 + 1 + 1 = 8/10. A randomized lift test is the best fit for the bounded renewal question, but the 24 is illustrative until the real assignment, uncertainty, lag, cost, and leakage checks pass. A broader budget shift would need additional evidence, such as calibrated MMM or further experiments.

Assumptions: treatment and holdout units are mutually exclusive, assignment is not affected by expected outcomes, qualified-lead definitions remain fixed, and the 30-day window captures relevant lag. Limitations: every number is invented for this example, no uncertainty interval or power calculation is shown, and the rate difference is not causal evidence for any real campaign.

Sensitivity before the renewal decision

The headline 2.0 percentage-point difference depends on the outcome definition and a clean holdout. Recalculate the same simulated rows when qualification rules, leakage findings, or lag handling change; do not preserve the headline by changing the denominator.

Scenario	Treatment rate	Holdout rate	Difference	Illustrative incremental leads	Renewal threshold
Base simulated counts	96 / 1,200 = 8.0%	18 / 300 = 6.0%	2.0 pp	1,200 x 0.020 = 24	Passes 15
Stricter qualification removes 12 treatment leads	84 / 1,200 = 7.0%	18 / 300 = 6.0%	1.0 pp	1,200 x 0.010 = 12	Fails 15
Holdout leakage adds 4 apparent holdout leads	96 / 1,200 = 8.0%	22 / 300 = 7.3%	0.7 pp	1,200 x 0.0067 = 8	Fails 15
Only 90 treatment leads meet the approved outcome	90 / 1,200 = 7.5%	18 / 300 = 6.0%	1.5 pp	1,200 x 0.015 = 18	Passes 15

Decision rule: (treatment rate - holdout rate) x treatment denominator

Under these invented counts, the renewal decision changes when the approved outcome definition removes enough treatment leads or when holdout leakage raises the comparison rate. That makes qualification stability and leakage checks decision-critical, not optional footnotes.

All rows are simulated and rounded for readability; 0.7 percentage points is displayed from 0.6667 percentage points. The table does not show uncertainty, statistical power, cost, or conversion lag, so it cannot establish that any scenario would renew a real campaign.

Worked decision board converting a weak before-after result into comparison checks, a controlled experiment recommendation, and bounded action rules. — A weak before-after result can still be useful if it is converted into the right next method. The review moves from the observed result, to comparison quality, to method fit, to a bounded action such as test, hold, scale, or repair.

Fast diagnostic

If the report answers "what happened after launch," ask for the comparison that shows what would have happened anyway.
If the report answers "who converted," ask whether those people were already more likely to convert before exposure.
If the report answers "which channel ranked highest," ask how uncertainty and calibration changed the ranking.
If the report answers "what people remembered," ask whether the survey population is comparable and whether the outcome matches the decision.
If the report answers "which placement got attention," ask what business decision attention is supposed to improve.

Takeaway

Choose the method after the decision is clear, not after the dashboard is available. A weaker method honestly labeled is more useful than a stronger-looking method used outside its lane.

Topic routes

Open the broader method route.

Use these routes when method choice leads into a campaign readout, a stronger test plan, or a worked failure mode.

Method laneSearch method-choice guidesCompare tests, MMM, attribution, brand lift, attention, baselines, and uncertainty. Readout routeAudit the finished reportMove from method fit into delivery, outcome quality, comparison strength, and decision language. Failure modesStudy worked examplesUse selection, attribution, survey, targeting, and geo-test cases before generalizing a result.

Decision routes

Related methods by decision

Use the reader's next decision to move from the selector into the right evidence check, rather than choosing the next page by method name alone.

Budget thresholdMatch action to evidence strengthUse when a result is being used to renew, shift, scale, cut, or add a stronger test. Uncertain estimateCheck the range before actingUse when a point estimate needs intervals, thresholds, or sensitivity before a decision. Baseline qualityTest the comparison firstUse when before-after, matched-period, or benchmark evidence may be carrying too much weight. Planning modelReview the MMM readoutUse when modeled contribution is shaping channel plans, response curves, or future assumptions.

Keep reading

Choose the next guide

Move from method choice into the budget decision, test plan, or readout check that matches the question.

Budget actionUse the evidence ladderDecide whether the result supports repair, renewal, budget movement, or a stronger test. Causal designWrite the lift-test planTurn a counterfactual question into assignment, outcome, leakage, and readout rules. Report reviewAudit the campaign readoutCheck whether the final report keeps delivery, response, quality, and lift language separate.