Readable worked examples of how advertising measurement can go wrong: the business model, the tempting metric, the hidden bias, and the better test design.
Use these routes when a worked example has made the risk visible and the next step should improve a claim, method, readout, or test plan.
Use the library to identify a likely failure mode, then pressure-test the claim and choose the measurement design that would answer the real business question.
Use these clusters to move from a worked example to the guide that tests the next claim, report, or campaign readout.
Search and retargeting cases show how commercial intent, cart return behavior, and short conversion lag can make attribution look like lift.
Card-linked, cash-back, and receipt-reward cases show how offer claiming, transaction history, and brand affinity can bias the measured group.
The geo-lift case shows why market choice, pre-period trend, seasonality, and comparison rules need scrutiny before a post-launch gap becomes a media claim.
Brand-lift and lead-generation cases show how respondent mix, page behavior, and downstream quality can turn a proxy metric into an overstated outcome.
Each case is useful only if it improves the next review. Use this map to move from the tempting read to the checklist that tests the next claim.
| Case signal | Risk to test next | Use first | Decision it improves |
|---|---|---|---|
| Search or retargeting gets credit near conversion. | Prior intent, short conversion lag, and path credit being described as lift. | Attribution window checklist | Whether to treat the readout as descriptive path reporting or ask for incrementality evidence. |
| Offer claimers, card-linked audiences, or receipt submitters outperform. | Treatment reveals purchase intent, loyalty, category engagement, or selection that existed before exposure. | Audience selection checklist | Whether the result supports a broader audience claim or only describes a high-intent group. |
| A geo or market test shows a large post-launch gap. | Market selection, seasonality, pre-period trend imbalance, and comparison rules chosen after the fact. | Geo lift design checklist | Whether the market readout can support causal language or needs a cleaner matched-market plan. |
| A brand-lift readout shows survey movement. | Exposed/control recruitment imbalance, placement attention, sample quality, and proxy outcome inflation. | Brand lift readout checklist | Whether the result supports perception language, sales language, or only directional learning. |
| A lead-gen campaign produces more forms or matched pipeline. | Page behavior, form quality, lead status, sales follow-up, and matchback selection. | Landing page and lead quality checklist | Whether the readout supports lead-quality work, campaign repair, or a stronger test. |
| One case is being used as a rule for future buying. | Transfer conditions, eligible universe, comparison quality, and whether the outcome matches the next decision. | Case-study generalizability checklist | Whether the lesson can travel, or should remain a bounded warning about one measurement design. |
Use these answers to choose the right worked example, then move from the example to a checklist that can improve the next claim or campaign readout.
Start with the search-intent and retargeting cases. They show how commercial intent, cart-return behavior, and short conversion lag can make path credit look like incremental lift unless a holdout or suppression design tests the counterfactual.
Watch for offer claiming, receipt submission, card-linked eligibility, loyalty history, category visits, or any step that reveals higher purchase intent before the measured treatment. Those signals call for an audience-selection review before broad lift language.
Use the geo-lift case when treated markets had stronger baseline demand, seasonal timing, cleaner inventory, or post-launch exclusions that may explain the gap. The next step is to inspect pre-period fit and comparison-market rules before using causal language.
Treat survey movement and form fills as proxy signals until respondent mix, page behavior, lead quality, and downstream acceptance are checked. The case should move the review toward outcome quality, not stronger claims than the proxy can support.
Use the case to name a risk and choose the next checklist. Do not treat the example as proof that every similar campaign fails; check whether the audience, measurement window, outcome, comparison group, and decision context match the next situation.
A useful case should change the readout question: what comparison group is credible, which denominator matters, whether the outcome is valuable, and what language the evidence can support. Convert that into a QA note before renewing or scaling the campaign.
The exposed survey respondents are recruited from higher-attention placements and more category-involved sessions, while the control respondents come from broader lower-intent inventory, so respondent mix is mistaken for campaign-caused persuasion.
Naive: 20.5 pts Adjusted: 0.9 pts
Read case studyTargeting uses recent category behavior and purchase histories. The measured treatment group therefore contains shoppers with a higher baseline chance of buying even without the offer.
Naive: 14.8 pts Adjusted: 1.6 pts
Read case studyThe test group had to claim an offer before buying, while the control group did not complete an equivalent intent-revealing step. Claiming the offer is partly a measure of pre-existing purchase intent.
Naive: 26.0 pts Adjusted: 2.0 pts
Read case studyThe treated markets already had stronger category momentum, higher baseline demand, and more favorable timing before the campaign, so the readout treats market selection and seasonality as incremental media impact.
Naive: 20.0 pts Adjusted: 1.2 pts
Read case studyThe measured group is enriched for visitors who already had stronger buying intent before the campaign, so form fills and matched pipeline are treated as incremental demand.
Naive: 24.1 pts Adjusted: 1.2 pts
Read case studyUsers who clip, activate, or submit qualifying receipts are already more engaged with the category and more likely to buy the brand. The action required for treatment reveals intent that the control group never had to reveal.
Naive: 15.8 pts Adjusted: 1.1 pts
Read case studyThe treatment group is enriched for people who recently browsed, compared, saved, or abandoned a cart, so the campaign is credited for conversions that many users were already likely to complete.
Naive: 29.6 pts Adjusted: 0.9 pts
Read case studyThe treatment group is enriched for shoppers who typed commercial or branded queries, so last-click conversion credit captures demand that already existed before the ad impression.
Naive: 28.5 pts Adjusted: 0.6 pts
Read case study