Source quality

Source role and incentive map for media claims

Organization byline: Measurement Press editorial desk. This page does not list an individual author or reviewer. Read the editorial method and corrections policy.

About 11 min read

On this page

A source can be honest, useful, and still too interested, too far from the record, or too narrow for the claim it is being asked to support. The reader's job is to identify what role each source is playing before accepting the story's frame.

This map works for news stories, research reports, dashboards, vendor claims, and advertising measurement readouts. It does not label sources as good or bad. It separates access, expertise, incentive, and evidence so a reader can keep confidence proportional to the source trail.

A practical example: a campaign vendor sends a report saying exposed households converted 18% more often. That sentence may be useful, but it does not tell you whether the vendor measured a randomized comparison, matched people who were already likely to buy, or counted platform-visible activity as a sale. Map the vendor's role, the advertiser's outcome record, the platform's exposure rule, and the missing comparison before the number changes a budget.

Source role and incentive decision map with source role, date, audience, direction, evidence, and approval cards connected to one report. — The source-role map asks what each source contributes before the conclusion is approved. Role, timing, affected audience, direction of interest, and evidence strength should all remain visible beside the claim.

The source role map

Start by assigning each quoted or cited source a role. The role tells you what the source can help establish and what extra checks the claim still needs.

Source role	Useful for	Ask before trusting	Weak use
Primary record owner	Showing what was filed, counted, published, logged, or formally decided.	Does the story show the record scope, date, definition, and denominator?	Using the record's existence as proof of the broadest interpretation.
Direct participant	Confirming firsthand experience, sequence, statements, and incentives.	Which parts are observed facts, memory, interpretation, or advocacy?	Treating one participant's account as the full population or final cause.
Affected party	Explaining lived impact, operational details, and missing practical context.	Is the experience representative, illustrative, or an outlier?	Letting the strongest anecdote stand in for the denominator.
Independent subject expert	Explaining mechanism, method, precedent, and uncertainty.	Is the expertise directly related to the claim, and is the evidence basis visible?	Using authority to replace the primary record or measurement method.
Research producer	Explaining sample, model, survey, or analysis choices.	Who funded the work, who owns the data, and what limitations are disclosed?	Treating a method-bearing report as broader proof than the method supports.
Sponsor or vendor	Clarifying product design, business logic, study objective, or reported metric definition.	Does the article separate sponsor interest from method evidence?	Letting the party with the strongest stake supply both claim and conclusion.
Trade group or advocacy organization	Finding stakeholder positions, policy arguments, member data, and issue framing.	Whose interests are represented, and what counter-source should be visible?	Presenting an organized position as neutral consensus.
Platform or data owner	Defining eligible inventory, users, impressions, outcomes, match rules, and reporting limits.	What cannot be independently observed, audited, or compared?	Converting platform-visible activity into causal impact language.
Anonymous or background source	Providing narrow factual leads, process context, or documents that can be checked another way.	Why is anonymity needed, and does the article corroborate the claim?	Letting unnamed interpretation carry motive, scale, or causal judgment.
Secondary analyst or commentator	Orienting readers, identifying patterns, and comparing public material.	Does the analysis link back to inspectable records and method notes?	Replacing the original record with a confident summary.

The incentive pass

Incentive is not a dismissal. It is context a reader needs before deciding whether a source is carrying facts, interpretation, or a desired outcome.

Financial incentive

The source benefits if a product, category, service, investment, or campaign is viewed favorably. Look for independent denominators, method notes, and outcome definitions.

Reputational incentive

The source benefits if a prior decision, forecast, investigation, strategy, or public position appears correct. Look for disconfirming context and error boundaries.

Political or advocacy incentive

The source benefits if readers accept a policy frame, moral frame, or urgency frame. Look for definitions, base rates, omitted groups, and response parity.

Access incentive

The source has documents, logs, data, or insider context other sources lack. Look for corroboration, narrow wording, and whether access is being upgraded into conclusion.

Method incentive

The source built the metric, model, survey, or dashboard being discussed. Look for validation, uncertainty, comparison rules, and where the method cannot observe the counterfactual.

Four-panel incentive pass showing financial, reputational, advocacy, and method or access incentives alongside interest, evidence, and limits. — An incentive explains what a source may emphasize or omit; it does not prove dishonesty. Keep the interest visible, then test the evidence and the limit separately.

What each role can support

Claim type	Strong source mix	Missing-context risk
Something happened	Primary record plus direct participant or official statement.	The event is real, but scale, representativeness, or cause is still unsupported.
A pattern is increasing	Time series, denominator, method note, and comparison period chosen before interpretation.	A convenient start date makes ordinary variation feel exceptional.
A group believes something	Survey disclosure with sample source, wording, field dates, weighting, and uncertainty.	A stakeholder quote or small sample is treated as public consensus.
A product or campaign worked	Decision-relevant outcome, comparison rule, exposure quality, holdout or model diagnostics, and uncertainty.	Selection, prior intent, attribution, or matched activity is promoted into lift.
A policy or business decision caused a result	Timeline, mechanism, counterfactual, denominator, and alternative explanations.	Timing is treated as causality because the source role fits the desired frame.
A report proves a category trend	Report method, data universe, sponsor role, base period, exclusions, and external benchmark.	Vendor or sponsor evidence is broadened beyond its own measured universe.

Standards that sharpen the map

This map is an editorial decision aid, not a legal test or a substitute for subject-matter review. The Society of Professional Journalists Code of Ethics is a useful journalism boundary because it connects source work to verification, source identification, access to source material, and disclosure of unavoidable conflicts. It does not certify a source or settle a disputed claim.

For public survey claims, the AAPOR disclosure standards are more specific: inspect the sponsor and researcher, population, recruitment, mode, field dates, sample size and precision, wording, weighting, quality procedures, and limitations. Missing disclosure narrows what the result can support; disclosure alone is not proof of representativeness.

For endorsements or sponsored promotional statements, the Federal Trade Commission’s endorsement guidance treats a relationship that could affect how people evaluate an endorsement as a material connection that should be disclosed. That is a disclosure boundary, not a general rule that an interested source is false.

What the checked standards establish: SPJ supports verifying information, using original sources where possible, identifying sources and motives, and providing relevant source material. AAPOR makes survey disclosure a method-and-limitations request, including sample, precision, weighting, processing, and response information where applicable. FTC guidance makes a material connection a reader-facing disclosure issue when it could affect the weight or credibility of an endorsement. None of these standards validates the simulated survey, proves representativeness, or turns an interested source into an independent one.

Map dimension	Reader inspection	Useful standard anchor
Proximity and accountability	Can the reader reach the original record, full report, transcript, or method note, and can the source be identified?	SPJ source access and verification
Survey completeness	Are the population, recruitment, dates, mode, sample size, instrument, weighting, quality checks, and limitations visible?	AAPOR disclosure standards
Incentive disclosure	Would a reasonable reader change the weight given to the claim if the sponsor, payment, employment, or other relationship were known?	FTC material-connection guidance

Source role to decision handoff

Use the map to create a concrete next request. The role names what a source can contribute; the evidence request tests the gap; the claim ceiling prevents a useful source from silently becoming the whole conclusion.

Source role	Request before the decision	Claim ceiling until it arrives
Primary record owner	Retrieve the original record, definition, date range, coverage note, revision history, and denominator used in the sentence.	What the record contains or reports; not the full universe, prevalence, or cause.
Research producer	Request the instrument, population, recruitment path, mode, field dates, sample bases, weighting, quality checks, and limitations.	What the disclosed respondents or analyzed units show; not representative opinion without a justified frame.
Sponsor or vendor	Separate the interested party's metric definition and source files from independent validation, comparison rules, and outcome records.	A sponsored or vendor-reported result can be described, but it does not become independent proof or lift.
Platform or data owner	Document eligibility, exposure or match rules, observable fields, exclusions, and what the platform cannot see or compare.	Platform-observed activity within its measured universe; not causal impact across unseen users or channels.
Affected party or advocate	Seek a specific response, counter-source, affected-group context, and the denominator that separates experience from scale.	A reported experience or position may add context; it does not establish prevalence, motive, or consensus alone.

Simulated Worked Example: a sponsored survey claim

This is a simulated example, not a real survey or a claim about any named organization. Assume a fictional release says, “62% of agency buyers plan to increase video spend next quarter.” The release identifies its sponsor and research producer, names agency buyers as the target population, gives an online mode, reports field dates, and says n=320. A fictional recruitment log shows 1,000 invitations and 320 completed responses. The release does not show recruitment rules, question wording, weighting, quality controls, or limitations.

Source role or check	What the simulated record shows	Decision effect
Sponsor or vendor	Named and financially interested in a favorable category interpretation.	Keep the interest visible; do not let the sponsor supply both evidence and conclusion.
Research producer	Named, but its instrument, recruitment, weighting, and quality procedures are not disclosed.	Method role is present; method audit remains incomplete.
Direct participants	320 respondents answered the online instrument.	Supports what those respondents said, not automatically what all agency buyers believe.
Primary record	The release gives 198 “increase” responses out of 320.	The result is inspectable as a respondent count, but the population frame is still unresolved.
Independent check	No outside benchmark, probability frame, or non-sponsor replication is supplied.	Do not upgrade a sponsor-released respondent result into a market-wide trend.

Reproducible arithmetic: the stated percentage is 198 / 320 = 61.875%, which rounds to 62%. The simulated response rate is 320 / 1,000 = 32.0%, but that rate describes the fictional invitation log, not representativeness. A six-check audit scores record scope 1/1, sponsor visibility 1/1, method visibility 1/1, respondent denominator 1/1, population generalizability 0/1, and independent confirmation 0/1: 4 / 6 = 66.7%. That score is a workflow diagnostic, not statistical confidence.

Decision: hold the market-wide claim. The defensible sentence is, “A sponsor-released online survey reported that 198 of 320 respondents planned to increase video spend, but the public release does not establish how that result represents all agency buyers.” Do not say that agency-buyer demand is rising until the population frame, recruitment, wording, weighting, quality controls, limitations, and an independent comparison are available.

Reproduction notes and limits: count a disclosure field only when the release or a directly linked method note states it; do not infer missing methods from the sponsor’s reputation. Re-score if those records are added. The simulated denominator is fixed at the respondent level, so it cannot establish prevalence, causation, future spending, or a market trend. A different recruitment frame, nonresponse pattern, weighting scheme, or question wording could materially change the conclusion.

Quote-balance checks

Balance is not just counting quotes. A story can quote several people and still give one role more evidentiary weight than the record supports.

Check whether the central claim is supported by records while the response is only summarized, or the reverse.
Separate fact witnesses from interpretation sources before comparing quote length.
Look for verb differences: "said," "claimed," "admitted," "warned," and "conceded" can tilt the reader before evidence appears.
Notice whether caveats appear near the claim or only after the strongest frame is already established.
Ask whether the omitted source role is obvious: data owner, affected party, method expert, regulator, sponsor, vendor, or counterfactual comparison.

Advertising measurement readouts

The same role map applies when a campaign result is being used as evidence. A platform, vendor, agency, brand, publisher, and analyst can each be useful while still seeing only part of the evidence trail.

Measurement source	Can usually show	Still needs
Publisher or inventory owner	Placement IDs, context, creative delivery, eligible impressions, ad sizes, and page-level package logic.	Advertiser-side outcomes, comparison design, and independent lift evidence.
Ad server or platform	Served impressions, clicks, viewability fields, frequency, segments, and platform-visible conversions.	Outcome quality, unmatched users, cross-channel exposure, and incrementality.
Measurement vendor	Method note, match rule, model output, survey result, or attribution window.	Validation, uncertainty, leakage checks, and decision limits.
Advertiser or CRM owner	Lead status, sales stage, revenue fields, follow-up coverage, and duplicate handling.	Media exposure quality, audience selection, and counterfactual comparison.
Agency or analyst	Campaign interpretation, budget context, reporting structure, and next-action recommendation.	Clear separation between observed results, attributed activity, and causal claims.

Worked advertising measurement readout with publisher, ad platform, measurement vendor, advertiser CRM, and analyst evidence feeding observed, attributed, causal, and decision language. — A readout becomes decision-ready only when its evidence sources and language level are separated. Observed, attributed, and causal are different claims, not interchangeable reporting labels.

Fast workflow

Four-step source mapping workflow: name the claim, assign each source a role, mark the strongest incentive, and find the missing role. — Run the four steps before debating whether a story sounds balanced. The missing role often reveals which conclusion is being asked to carry more weight than its evidence can support.

1. Name the claim.

Write the article, report, or readout claim as one sentence. Identify whether it is factual, comparative, causal, predictive, or normative.

2. List every source role.

For each quote, link, chart, report, dataset, and dashboard, assign one role from the map. Do not let the same source quietly play every role.

3. Mark the strongest incentive.

Financial, reputational, advocacy, access, and method incentives should be visible beside the claim they help support.

4. Find the missing role.

Ask which source role would most likely weaken, narrow, or clarify the frame. If that role is absent, confidence should drop.

5. Calibrate the language.

Use "shows," "suggests," "is consistent with," or "does not establish" according to the source trail, not according to how persuasive the narrative feels.

Use with

Pair this map with the source quality scorecard when evidence strength is unclear, the quote weight and response standard checklist when balance is the issue, the sponsored research and vendor report checklist when a report has an interested sponsor, and the campaign readout QA checklist when a measurement source is being asked to support a budget decision.

Keep reading

Choose the next claim check

Move from this page into source evaluation, confidence scoring, and claim-language review before accepting a public frame.

Evaluation deskReview the evidence trailUse the source and vendor evaluation desk to inspect incentives, proximity, denominators, and limits. Confidence rubricSet the evidence levelDecide whether the support is decision-grade, strong but bounded, directional, weak, or unsupported. TriangulationCheck independent supportSeparate repeated citation loops from independent confirmation, counter-sources, and method-bearing records.