Evidence standard
Causal claim review protocol
Causal language is where weak evidence often becomes expensive. A story says an event caused a change, a dashboard says a campaign drove sales, or a report says a tactic improved an outcome. The reader needs a fast way to decide whether the verb has earned that confidence.
This protocol is for any claim that uses words like caused, drove, led to, changed, increased, reduced, prevented, influenced, or worked. It helps separate observed sequence from causal evidence before the claim becomes a headline, meeting takeaway, renewal argument, or budget recommendation.
The six-question pass
1. What exactly changed?Name the outcome, population, time window, geography, and unit of analysis. A causal claim is not reviewable until the claimed change is specific enough to inspect.
2. What is the proposed cause?Separate one event, policy, campaign, message, product change, or source action from the background conditions around it. If the cause is vague, the evidence standard is already lower.
3. What would have happened anyway?Name the counterfactual: prior trend, peer group, holdout, matched market, untreated users, comparable records, model baseline, or survey control. If no counterfactual is visible, the claim should not sound settled.
4. Was the comparison chosen before results were visible?Post-hoc comparisons can still teach, but they are weaker than planned comparisons. Watch for convenient chart windows, selective markets, cherry-picked segments, and omitted failed outcomes.
5. What else changed at the same time?List concurrent events, seasonality, pricing, distribution, creative, product changes, reporting changes, data-quality shifts, eligibility changes, and audience mix changes. Each plausible alternate explanation should narrow the claim.
6. What language can the evidence support?Choose the verb after the design is visible. "Followed," "correlates with," "is consistent with," "suggests," "estimated lift," and "supports a causal read" are different claims.
Claim-type matrix
| Claim type | Strongest first check | Common overclaim | Safer readout language |
|---|---|---|---|
| News timeline | Prior state, timestamped records, mechanism, lag window, and concurrent events. | Sequence is treated as cause. | "The event preceded the change; the record should show whether the change was already underway." |
| Before-and-after chart | Full source series, denominator, chart window, seasonality, peer comparison, and revision risk. | A visible break is treated as proof of impact. | "The series changed after the event, but the comparison is not yet strong enough for causal language." |
| Public-record count | Record universe, inclusion rules, missing cases, rate base, and comparable period. | A count change is treated as population change. | "The reported count changed in this record universe; the rate base determines how broad the claim can be." |
| Campaign attribution | Exposure rule, attribution window, prior intent, comparison group, and credited outcome definition. | Credited conversions are treated as incremental conversions. | "The report credits observed outcomes to the campaign; lift still needs a protected comparison." |
| Lift test | Assignment, holdout protection, leakage, primary outcome, minimum detectable effect, and uncertainty. | A noisy or contaminated test is treated as decisive. | "The test estimates lift under this design and uncertainty range." |
| Model output | Controls, priors, calibration evidence, sensitivity, response curves, and omitted drivers. | Predictive fit is treated as causal proof. | "The model supports a planning range if the assumptions and calibration evidence hold." |
| Survey or brand study | Population, sample source, exposed/control balance, wording, field dates, and respondent quality. | Respondent movement is treated as market or sales movement. | "The survey reports a difference among respondents under this field design." |
Downgrade triggers
Move a causal claim down at least one confidence level when one of these conditions is present. Move it down more than one level when several appear together.
- The evidence shows only what happened after launch, publication, announcement, or exposure.
- The comparison group was chosen after the favorable result was visible.
- The affected group was already different before the claimed cause occurred.
- The denominator, starting level, or eligible population is hidden.
- The outcome changed at the same time as pricing, seasonality, product availability, measurement rules, or audience mix.
- The reported result is a proxy for the decision outcome: attention instead of sales, recall instead of persuasion, clicks instead of lift, or quoted reaction instead of evidence.
- The uncertainty interval, subgroup base, sample source, or exclusion rule is missing.
Evidence request menu
| If the claim depends on | Ask for | Decision rule |
|---|---|---|
| Chronology | Timestamped records, prior-state evidence, lag expectation, and concurrent-event list. | Use causal wording only if the mechanism and timing survive alternate explanations. |
| A trend chart | Underlying data, full period, denominator, seasonality check, peer trend, and revision status. | Do not let a chosen window carry the whole claim. |
| A campaign readout | Holdout, matched comparison, mature outcome window, exposure logs, leakage check, and pre-period intent. | Keep attribution language separate from lift language. |
| A model | Model specification, controls, priors, calibration tests, sensitivity checks, and uncertainty ranges. | Use model output as a bounded estimate, not a standalone proof. |
| A survey | Population, sample source, field dates, exact wording, weighting, subgroup bases, and control balance. | Keep the conclusion inside the respondent universe and measured construct. |
| A public record | Record owner, inclusion rules, missing universe, rate base, comparable period, and revision rules. | State what the record shows before claiming what actually changed in the wider world. |
Decision note template
Use this short note before forwarding, publishing, renewing, or escalating a causal claim.
Claim:Write the exact sentence and mark the causal verb.
Observed fact:State what the source clearly shows without interpretation.
Counterfactual:Name the comparison used, or state that no credible comparison is visible.
Alternate explanations:List the strongest timing, selection, denominator, mix, measurement, or incentive explanations.
Confidence level:Choose decision-grade, strong but bounded, directional, weak signal, or unsupported frame.
Allowed wording:Write the strongest sentence the evidence can support today.
Use it with the rest of the library
For media claims, pair this protocol with the timeline and event-order framing checklist, before-and-after trend checklist, public records and denominator checklist, media claim audit worksheet, and evidence-to-claim language matrix.
For advertising measurement, pair it with the measurement method selector, incrementality test plan template, randomized lift test readout checklist, MMM causal validity checklist, and campaign readout QA checklist.
Takeaway
Causal wording should be earned, not inherited from a chart, quote, dashboard, or vendor summary. When the counterfactual is weak, keep the sentence descriptive. When the design is strong, state the limits that make the causal claim useful.