Campaign measurement
Identity matchback measurement checklist
A matchback report can feel unusually concrete: exposed households, matched customers, offline conversions, revenue, lead status, or clean-room overlap. The hard question is whether the matched outcome was caused by the campaign or only found after the campaign.
Use this checklist when a publisher, platform, retailer, agency, or data partner reports matched conversions, matched sales, store visits, CRM outcomes, household reach, or clean-room overlap. The goal is to keep identity coverage, outcome quality, and causal evidence in separate lanes.
Start with what was matched
The same phrase can describe very different evidence. Before reading the result, identify the entity, source, and window that created the match.
| Matchback result | What it can show | What it cannot prove alone |
|---|---|---|
| Audience match rate | How much of a buyer file, prospect list, or exposed population could be resolved to the reporting system. | That the matched group represents the unmatched group. |
| Household or device reach | Observed delivery against an identity graph, household model, device set, or account universe. | That every reached person saw the ad or that reach created the outcome. |
| Matched offline conversions | Observed customer, transaction, lead, or store-visit activity tied to matched identifiers inside a window. | That those outcomes would not have happened without exposure. |
| Clean-room overlap | How two permissioned datasets overlap under the selected join keys and privacy thresholds. | That overlap is incremental demand or that suppressed rows behave like visible rows. |
| Matched revenue or pipeline | Reported value from matched accounts, customers, leads, or orders. | That campaign exposure created the revenue, pipeline, or account movement. |
Minimum disclosure packet
Ask for the packet before debating the headline metric. A matchback result is only useful when the matching rule and the missing universe are visible.
Identity unitName whether the match is at the person, browser, device, household, account, location, loyalty ID, email hash, phone hash, postal address, or company-domain level. Do not let these units blur together in the readout.
Eligible universeState who could be matched, who could be exposed, who could convert, and which records were excluded before matching. A high match rate against a narrowed universe can still miss the hardest-to-measure customers.
Match methodDocument deterministic joins, modeled joins, recency rules, confidence thresholds, deduplication, householding, and fallback logic. A result using several identity steps should show each step separately.
Outcome sourceIdentify the system of record for purchases, visits, leads, qualified leads, pipeline stages, renewals, or revenue. The outcome source should define reversals, cancellations, returns, repeat orders, and duplicate records.
Time windowsShow exposure window, lookback window, attribution window, conversion window, data-lag allowance, and any pre-period used to detect prior intent.
Comparison ruleName the holdout, geo baseline, matched control, prior-period rule, model baseline, or explicit statement that no causal comparison exists.
Bias checks
Matchback reports often look precise while selecting the customers who were easiest to identify, easiest to reach, or already closest to action.
| Risk | QA question | Why it matters |
|---|---|---|
| Match-rate selection | Do matched and unmatched records differ by prior purchase, geography, device, channel, account size, or customer age? | Matched outcomes can overstate campaign quality when higher-value customers are easier to resolve. |
| Prior intent | Did matched converters already search, visit, buy, request content, enter CRM, or receive sales contact before exposure? | The match may be finding demand already in motion. |
| Exposure leakage | Could control users see the campaign through another device, household member, channel, publisher, or campaign? | Leakage reduces the meaning of exposed-control comparisons. |
| Outcome duplication | Can one person, household, account, or order appear multiple times across systems? | Duplicate outcomes can inflate both conversion count and revenue value. |
| Window shopping | Were attribution windows, lookbacks, or lag rules chosen after the result was visible? | Flexible windows can turn ordinary timing into apparent performance. |
| Suppression and thresholds | Are small cells, privacy thresholds, unmatched rows, and unjoinable outcomes reported as missing rather than zero? | Hidden rows can change the denominator and the direction of the result. |
Matchback quality score
Score the readout before the headline number is used in a decision. A strong matchback review does not ask whether the joined records look impressive. It asks which weak lane caps the conclusion.
| Review lane | Green | Yellow | Red |
|---|---|---|---|
| Coverage | Eligible exposure, control, and outcome universes are named; unmatched and suppressed rows are reported separately. | Match rate is visible, but the unmatched population is described only in broad terms. | The report shows only matched outcomes and hides who could not be joined. |
| Identity rule | Join keys, householding, confidence thresholds, deduplication, and fallback logic are documented step by step. | The vendor names deterministic or modeled matching but does not show step-level loss. | Identity matching is treated as a black box or a single blended rate. |
| Outcome quality | Outcome source, duplicate handling, reversals, lead status, returns, and lag rules are tied to the business decision. | The outcome is plausible, but maturity, duplicates, or status changes still need closeout. | Raw conversions, visits, leads, or revenue are counted without eligibility or quality rules. |
| Prior intent | Matched converters and nonconverters are checked for pre-period search, purchase, CRM, store, account, or sales activity. | Some pre-period behavior is shown, but important intent signals are missing. | The readout begins at launch and ignores whether matched converters were already close to action. |
| Comparison strength | A planned holdout, geo baseline, balanced matched control, or other credible counterfactual protects the causal language. | The comparison is directional and useful, but leakage, balance, or window choice limits the claim. | Matched outcomes are presented as lift without a credible baseline. |
Readout ladder
Use language that matches the design. Matched outcomes can be valuable operational evidence without becoming a lift claim.
| Evidence available | Careful wording | Overclaim to avoid |
|---|---|---|
| Match rate only | The reporting system resolved this share of the eligible records under the stated match rule. | The campaign reached this share of the whole market. |
| Matched outcomes with no baseline | Matched users produced observed outcomes inside the reporting window. | The campaign caused those outcomes. |
| Prior-period comparison | Matched outcomes were higher or lower than the selected prior period, with known context changes. | The campaign caused the full change. |
| Matched control | Observed outcomes differed from a defined comparison group, subject to balance and leakage limits. | The result is equivalent to random assignment. |
| Designed holdout or geo test | The campaign produced measured lift for this eligible population, match method, window, and uncertainty range. | The same lift should apply to every unmatched user, future flight, or channel. |
Worked downgrade example
A campaign report shows 1,800 matched purchasers and a high matched revenue total after exposure. The report also shows that only loyalty-account buyers could be joined, recent category visitors were overrepresented in the exposed group, returns are not yet settled, and the comparison group is a broad unexposed audience rather than a planned holdout.
The clean readout is not "the campaign drove 1,800 purchases." The careful version is: "The campaign reached a joinable loyalty-account population that produced 1,800 observed purchases inside the reporting window. Because prior intent and comparison balance are weak, this supports outcome follow-up and audience-quality review, not an incrementality claim."
The repair is practical: rerun the readout after return and lead-status windows close, split matched and unmatched populations, compare pre-period purchase behavior, and set a holdout or geo baseline before the next campaign. If those repairs are not possible, keep the result in the observed-outcome lane.
Questions for the vendor call
- What entity was matched, and what entity was counted as the outcome?
- What share of eligible exposed users, control users, and outcomes could not be matched?
- How do matched and unmatched records differ before the campaign starts?
- Which joins are deterministic, which are modeled, and what confidence thresholds are applied?
- What window was set before launch, and which windows were inspected after launch?
- How are duplicate devices, households, accounts, repeat orders, returns, and lead-status updates handled?
- What comparison protects the readout from prior intent, seasonality, sales activity, and concurrent campaigns?
Pair with
Use this checklist with the campaign readout QA checklist for finished reports, the private marketplace measurement checklist before launch, the campaign KPI dictionary for metric language, the campaign status-window closeout checklist when match runs, duplicate cleanup, lead status, or offline outcome rows are still maturing, the comparison market and holdout planning guide when a comparison group is needed, and the source library for outcome and data-quality references.