Media framing

Disconfirming evidence checklist for media claims

Published June 1, 2026. Status: evergreen source page.

A strong frame should survive contact with the best evidence against it. Before accepting a media claim, name the fact, denominator, comparison, counter-source, or method limit that would weaken the story if it appeared.

This checklist is for readers, editors, analysts, and researchers reviewing public claims from articles, reports, dashboards, records, surveys, and campaign readouts. The goal is not reflexive doubt. The goal is to make confidence depend on what the claim has already survived.

The one-page disconfirmation pass

Check	Ask for	Why it matters	Downgrade when
Central claim	The single sentence the reader is being asked to believe.	Disconfirmation is impossible until the claim is specific enough to test.	The frame depends on vague words such as crisis, surge, backlash, many, or experts.
Primary source	The closest record, transcript, table, method note, data release, or report behind the claim.	A summary can be accurate, but it cannot replace the thing being summarized.	The story links commentary, screenshots, or excerpts while the underlying material remains hard to inspect.
Alternate denominator	The population, eligible cases, total contacts, prior-period level, or opportunity base that could change the practical meaning.	A true numerator can still create a false sense of scale.	The claim gives a count, percentage, or rate without showing what base would make it fair.
Fair comparison	The peer group, prior period, matched geography, expected baseline, or historical range chosen before the conclusion.	Most frames are built by making one comparison easy and another comparison hard.	The article switches comparison classes depending on which one makes the claim sound strongest.
Missing source role	The record owner, affected party, method expert, counterparty, data owner, or independent analyst most likely to narrow the claim.	Absence of a role can matter as much as imbalance among visible sources.	One interested source supplies both the fact pattern and the interpretation.
Alternative explanation	A plausible non-causal explanation: seasonality, backlog, reporting change, selection, regression to the mean, audience intent, or definition shift.	A claim can be directionally true while its implied cause is weak.	The article treats timing, sequence, or correlation as enough to support cause.
Scope limit	The geography, population, sample, platform, data universe, field dates, or measured inventory the evidence actually covers.	Frames often become misleading when a narrow source is written as a broad conclusion.	A local, sampled, platform-visible, or customer-only result becomes a category-wide claim.
Decision threshold	The missing fact that would materially change confidence, not just add minor nuance.	Readers need to know what would move the claim from strong to tentative.	Caveats are listed vaguely without saying whether they would change the conclusion.

Use by claim type

Claim type	Strongest disconfirming evidence to look for	Useful paired guide
Public-record count	A different record universe, excluded population, rate base, delayed filing batch, definition change, redaction, or no-responsive-records boundary.	Public records and denominator checklist
Before-and-after trend	A longer chart window, seasonal pattern, concurrent change, policy timing mismatch, revised series, peer trend, or expected baseline.	Before-and-after chart and trend checklist
Survey or poll headline	Exact wording, field dates, low subgroup base, nonresponse, sponsor effect, weighting choice, coverage gap, or alternate question order.	Survey and poll claim checklist
Quote-driven story	A missing direct participant, document, response request, quote role, rebuttal placement, or comparable evidence standard for the other side.	Quote weight and response standard checklist
Sponsored or vendor research	Sponsor role, customer-only data, sample selection, omitted benchmark, denominator mismatch, method limit, or unpublished negative slice.	Sponsored research and vendor report checklist
Campaign or measurement readout	Holdout leakage, audience selection, attribution window choice, matchback bias, weak outcome quality, noisy interval, or missing comparison rule.	Campaign readout QA checklist
Source-quality claim	A closer primary record, more complete method disclosure, independent source, uncertainty statement, or replicable citation path.	Source quality scorecard

Downgrade rules

No named weakener

If a critique cannot say what evidence would weaken it, the frame is probably closer to persuasion than evaluation. Lower confidence until the claim has a clear test.

Same-source loop

If the central claim, denominator, comparison, and interpretation all come from the same interested source, treat the result as useful input rather than settled evidence.

Numerator-only pressure

If the article relies on a count, anecdote stack, percent increase, or superlative without a base, ask whether a rate, population, workload, or eligible universe would change the frame.

Post-hoc comparison

If the comparison appears chosen after the result is known, treat it as exploratory. Strong language needs a comparison rule that was fair before the conclusion was visible.

Proxy outcome upgrade

If a proxy such as attention, clicks, recall, matchbacks, complaints, responses, or filings is written as proof of the broader outcome, keep the claim inside the proxy.

Meeting worksheet prompts

What exact sentence are we testing?
What is the closest primary source, and what source is still missing?
What denominator or comparison would make the claim smaller, larger, or more ordinary?
Which source role would be most likely to narrow the claim?
What single fact would move this from strong confidence to modest confidence?
What wording keeps the conclusion inside what survived the disconfirmation pass?

Safer claim language

When the weakener is missing	Use language like	Avoid language like
The denominator is unclear.	"The available count shows reported cases, but the relevant rate base is not yet visible."	"The problem is widespread."
The comparison was not pre-specified.	"The comparison is suggestive and should be treated as exploratory."	"The comparison proves the claim."
The source role is narrow.	"This source can speak to its observed portion of the record."	"This source settles the broader question."
The causal path is not tested.	"The timing is consistent with the claim but does not establish cause."	"The event caused the result."
The method is proxy-based.	"The metric describes observed activity under this measurement rule."	"The metric proves real-world impact."

Use with

Pair this checklist with the media claim audit worksheet for a fast first pass, the source role and incentive map when a source may be carrying too much of the frame, the denominator framing examples when the base is unclear, and the evidence-to-claim language matrix when the final wording needs to match the evidence.

Takeaway

The best readers do not only ask whether a claim has evidence. They ask what evidence would make the claim weaker, narrower, or less causal. A frame that can answer that question deserves more confidence than one that cannot.