Search advertising
The Search Campaign That Confused Intent Capture With Incrementality
A realistic paid-search measurement scenario showing how commercial query intent can turn captured demand into an overstated incrementality and ROAS claim.
Archetype: Search advertising platform selling sponsored results beside high-intent queries
Bias mechanism: The treatment group is enriched for shoppers who typed commercial or branded queries, so last-click conversion credit captures demand that already existed before the ad impression.
Business model pressure
Advertisers pay for clicks or conversions associated with sponsored search placements. The platform has a strong reason to show that paid clicks produce profitable conversions.
Advertiser proof claim
A dashboard reports high ROAS from paid search clicks, but the counterfactual question is how many conversions would have happened through organic results, direct navigation, or another unpaid path.
Statistical result
| Metric | Naive read | Stratified read | Modeled benchmark |
|---|---|---|---|
| Conversion-rate lift | 28.5 pts | 0.6 pts | 0.6 pts |
| Attributed incremental conversions | 22,862 | 492 | 481 |
| Attributed conversion value | $2,194,732 | $47,227 | $46,214 |
| Attributed value after media cost | 4.57x | 0.10x | 0.10x |
The paid-search dashboard read is 46.5x larger than the query-intent counterfactual estimate in this worked example.
Readout audit questions
Use this pass before the case becomes a budget argument, buyer proof point, or channel lesson. The goal is to separate what the dashboard observed from what the campaign plausibly changed.
| Check | Question to ask | Evidence in this case | Safer claim boundary |
|---|---|---|---|
| Credited outcome | What is the report counting as campaign impact? | Separate attributed outcomes from incremental outcomes. In this case, the naive lift is 28.5 pts while the stratified read is 0.6 pts. | Treat path, click, form, survey, or matched-outcome reporting as descriptive until the counterfactual is visible. |
| Comparison group | Were treatment and control groups comparable before the campaign? | The treatment group is enriched for shoppers who typed commercial or branded queries, so last-click conversion credit captures demand that already existed before the ad impression. | Ask for a protected holdout, balanced market, suppression test, or matched comparison before using causal verbs. |
| Budget threshold | Would the decision survive the adjusted read? | The naive read is 46.5x larger than the stratified estimate, and adjusted ROAS is 0.10x after media cost. | Use the result for diagnosis, repair, or test planning unless the adjusted evidence clears the decision threshold. |
| Next evidence | What would change the conclusion? | Design a cleaner holdout or suppression test before treating captured intent as incremental demand. | Name the missing evidence request before turning one worked example into a general rule. |
For the next review, pair this case with Incrementality test plan template and the claim confidence rubric.
Search-intent decision logic
Before this case becomes proof that paid search created demand, read the result through the query universe. The useful question is whether comparable high-intent searchers would have converted without the sponsored result.
| Review point | Evidence to request | Decision consequence |
|---|---|---|
| Eligible query universe | The full set of brand, category, competitor, and generic commercial queries that could have shown a sponsored result, including non-clicked impressions and suppressed opportunities. | If the report starts only with ad clickers, describe the result as last-click attribution rather than incremental demand. |
| Intent balance | Pre-period site visits, brand familiarity, query wording, product specificity, basket urgency, repeat-customer status, and organic rank for sponsored-click and comparison sessions. | If sponsored-click sessions carry stronger purchase intent, remove causal lift and ROAS language until the comparison lane is balanced. |
| Fallback path | Organic search, direct navigation, saved-cart, email, affiliate, and store-locator paths available to the same query sessions when the ad is not shown or not clicked. | If the buyer would have had a clean unpaid route, treat the sponsored click as captured demand until suppression evidence says otherwise. |
| Timing and deduplication | Time from query to click, click to conversion, cross-device matching rules, repeat clicks, and whether multiple channels claimed the same order. | If the conversion window mainly captures normal shopping completion, use assisted-path language rather than created-demand language. |
| Claim boundary | A prewritten ladder separating impression coverage, click capture, attributed conversions, high-intent response, and randomized incrementality. | If renewal depends on causal proof, require query-level ad suppression, brand-term holdouts, or a matched high-intent counterfactual before using sales-lift or ROAS proof language. |
Worked downgrade
The headline version of this case says paid search produced a 28.5 point conversion-rate lift, 22,862 attributed incremental conversions, and $2,194,732 in attributed conversion value. That version sounds decisive because the paid click sits close to the order and the dashboard can match the conversion to the campaign window.
The weaker read appears when the query log shows that sponsored-click sessions were more likely to include brand terms, exact product names, prior site visits, saved carts, repeat customers, and organic paths that would have kept the shopper moving. After comparing sessions within similar commercial-intent strata, the estimated movement falls to 0.6 points and $47,227 in attributed value against $480,000 of media cost. The modeled benchmark is also 0.6 points.
The safer readout sentence is: sponsored search clicks captured high-intent shoppers and produced attributed orders, but the evidence does not prove incremental demand because the campaign selected query sessions that were already close to conversion. The next action is to rerun with query-level suppression, brand and category term separation, mature conversion windows, and a protected holdout before using lift or ROAS language.
The advertiser-facing story
The dashboard makes paid search look exceptionally efficient because conversions are measured after the shopper has already revealed commercial intent. The ad receives credit for an order path that may have completed through organic search, direct navigation, saved-cart behavior, or another unpaid reminder.
What broke
Treatment assignment is not random. Query intent, brand familiarity, prior site visits, organic result position, product urgency, and repeat-customer status all affect both the chance of clicking the ad and the chance of buying. Comparing sponsored-click sessions with broad non-click sessions makes intent look like lift.
Better design
Use randomized ad suppression, brand-term holdouts, query-level experiments, or matched high-intent comparison sessions. Report incremental conversions, not only attributed conversions, and separate brand, category, competitor, and generic queries before turning captured demand into budget proof.
Propensity-strata audit
The adjusted estimate compares sponsored-click sessions and comparison sessions within similar commercial-intent strata. That does not replace randomized ad suppression, but it shows how much of the headline conversion gap was carried by people who had already searched with purchase intent.
| Propensity stratum | Query-session records | Sponsored-click conversion rate | Query-holdout conversion rate | Within-stratum difference |
|---|---|---|---|---|
| 2 | 5,768 | 9.2% | 7.3% | 1.9 pts |
| 3 | 18,414 | 11.0% | 10.3% | 0.6 pts |
| 4 | 15,513 | 14.6% | 14.6% | 0.0 pts |
| 5 | 6,749 | 20.6% | 18.3% | 2.4 pts |
| 6 | 1,518 | 23.5% | 26.1% | -2.6 pts |
| 7 | 1,534 | 39.1% | 33.5% | 5.6 pts |
| 8 | 13,441 | 48.5% | 47.6% | 0.8 pts |
| 9 | 47,053 | 61.1% | 60.4% | 0.7 pts |
| 10 | 15,010 | 72.3% | 73.0% | -0.7 pts |
Takeaway
A strong paid-search readout should not stop at last-click attributed conversions. It should show the full eligible query universe, brand and category term mix, organic or direct fallback paths, suppression records, conversion-lag maturity, and how much of the result survives a protected high-intent counterfactual.
Move from this case to the next review.
Use the case to tighten claim language, choose a better measurement method, and review the next campaign readout without treating a worked example as a rule.