Lead generation measurement
The Lead-Gen Campaign That Counted Form Fillers As New Demand
A realistic measurement scenario showing how a useful advertising channel can still produce a misleading read when the comparison group is wrong.
Archetype: Lead-generation campaign measuring attributed form fills and CRM matchbacks
Bias mechanism: The measured group is enriched for visitors who already had stronger buying intent before the campaign, so form fills and matched pipeline are treated as incremental demand.
Business model pressure
A lead-generation program earns budget by showing that contextual placements, paid traffic, or sponsored resources produce efficient form fills and later sales-qualified activity. The advertiser wants evidence that the program created demand, not only captured people who were already researching a solution.
Advertiser proof claim
A dashboard reports low cost per lead, strong CRM matchback, and attractive pipeline value from attributed form fills, but the counterfactual question is how many qualified opportunities would have appeared through direct, organic, referral, or sales-nurture paths anyway.
Statistical result
| Metric | Naive read | Stratified read | Modeled benchmark |
|---|---|---|---|
| Conversion lift | 24.1 pts | 1.2 pts | 0.7 pts |
| Incremental conversions | 16,511 | 820 | 479 |
| Incremental revenue | $4,622,961 | $229,516 | $134,046 |
| ROAS after media cost | 8.81x | 0.44x | 0.26x |
The naive analysis is 20.1x larger than the stratified estimate in this worked example.
The advertiser-facing story
The campaign appears efficient because the report starts with visitors who clicked through to a high-intent page, downloaded a gated asset, or submitted a contact form. Later CRM matching gives the attributed group a convincing pipeline story, especially when the report compares those form fillers with a broad pool of site visitors or accounts.
What broke
Form completion is not only an outcome. It is also a signal that the visitor had a problem, budget, urgency, or prior familiarity before the measured touch. If the control group includes people who never showed comparable intent, the readout turns lead qualification and pre-existing demand into media-caused lift.
Better design
Define the eligible audience before launch, hold back comparable accounts or markets, keep lead-scoring and sales-acceptance rules fixed, and report sales-qualified leads or opportunity value against that protected comparison. A useful readout should separate traffic, form conversion, lead quality, follow-up coverage, and incremental pipeline.
Propensity-strata audit
The adjusted estimate compares treated and control users within similar treatment-propensity strata. That does not replace a randomized holdout, but it shows how much of the naive result was carried by who entered treatment.
| Propensity stratum | Users | Treated conversion | Control conversion | Within-stratum lift |
|---|---|---|---|---|
| 1 | 73 | 0.0% | 6.0% | -6.0 pts |
| 2 | 11,152 | 8.6% | 7.4% | 1.2 pts |
| 3 | 19,200 | 11.7% | 11.1% | 0.7 pts |
| 4 | 11,199 | 17.8% | 15.5% | 2.3 pts |
| 5 | 3,571 | 22.2% | 21.4% | 0.8 pts |
| 6 | 2,632 | 31.4% | 32.7% | -1.3 pts |
| 7 | 10,688 | 41.2% | 37.8% | 3.4 pts |
| 8 | 28,316 | 51.3% | 50.5% | 0.8 pts |
| 9 | 29,465 | 64.1% | 63.0% | 1.1 pts |
| 10 | 1,704 | 77.8% | 77.2% | 0.6 pts |
Takeaway
A strong advertiser report should not stop at attributed conversions. It should show how the comparison group was built, whether treatment users had stronger prior intent, and how much of the result survives a better counterfactual.
Use this case in a readout review
After reading the failure mode, move from diagnosis to a better question set with these practical tools.
- Claim confidence rubricScore whether the readout supports strong, qualified, or tentative claim language.
- Measurement method selectorChoose the method that best fits the question: holdout, matched market, MMM, survey readout, or QA.
- Landing page and lead quality measurement checklistLook beyond form fills to lead quality, sales acceptance, source mix, and downstream outcome evidence.