Incrementality

Where lift tests and brand studies quietly lose truth

Published May 29, 2026. Status: evergreen source page.

Lift tests are closer to causal evidence than attribution reports, but they are not magic. Their value depends on assignment, compliance, measurement, statistical power, and whether the result is generalized honestly.

Brand studies have a different problem: they measure perceptions through sampled survey responses. They can be useful, but the gap between a survey lift and business impact is often under-described.

Advertisement In-article programmatic unit.

Seven traps

1. Holdout leakage

The control group is only meaningful if it is actually protected from exposure. Cross-device exposure, shared households, retargeting, and overlapping campaigns can contaminate the contrast.

2. Platform-only outcome visibility

A platform can only measure what it can observe or match. Missing conversions, modeled conversions, and identity graph limits should be disclosed.

3. Underpowered tests

Small tests often produce wide intervals that are summarized as a single lift number. A non-zero point estimate is not the same as a reliable decision signal.

4. Short window generalization

A two-week test during a promotional burst may not support a quarterly budget shift. The time window is part of the claim.

5. Survey recruitment bias

Brand studies depend on who answers, when they answer, and whether exposed and control respondents are comparable after recruitment. See the brand lift survey recruitment case study for a worked example.

6. Proxy outcome confusion

Ad recall, awareness, consideration, and purchase are not interchangeable. A brand metric can move without proving profitable incremental sales.

7. Incentive opacity

When the platform selling media also measures lift, the report should disclose methods, exclusions, uncertainty, and any modeling assumptions.

What a strong report includes

  • Assignment method and holdout definition.
  • Primary outcome and decision threshold chosen before reading results.
  • Exposure compliance and contamination checks.
  • Power analysis or minimum detectable effect.
  • Confidence or credible intervals, not only percent lift.
  • Generalization limits: audience, channel, campaign, season, and creative.

Attention measurement note

The IAB, MRC, and CIMM attention measurement framework names multiple approaches to measuring attention and emphasizes consistency, transparency, validation, and disclosure. That is useful, but attention still needs to be connected to the decision being made. A high-attention impression is not automatically incremental business value.

Reference: IAB attention measurement framework.