Evaluation worksheet
Source and vendor evaluation worksheet
Use this page in editorial meetings, analytics reviews, vendor calls, and campaign readouts when a source or report is asking for more confidence than the evidence may support.
The worksheet turns a broad claim into a few inspectable pieces: the source trail, denominator, comparison class, causal design, uncertainty, and the fact that would most change the decision.
What the worksheet should make visible
A source review is strongest when it separates the persuasive sentence from the evidence beneath it. Before scoring, write down the exact claim, the decision it is meant to support, and the closest inspectable source. Then check whether the denominator, comparison class, source incentives, and uncertainty are visible enough for the decision being made.
| Pressure in the room | Field to expose first | Why it matters |
|---|---|---|
| A public claim sounds settled because several sources repeat it. | The original source trail and whether later citations add independent confirmation. | Repeated summaries can make one thin origin look like consensus. |
| A percentage, rate, or lift number is driving confidence. | The denominator, starting level, eligible universe, and excluded rows. | A strong-looking change can disappear when the base or comparison changes. |
| A vendor result is being used for budget, renewal, or scale language. | The counterfactual, assignment rule, outcome window, leakage checks, and uncertainty. | Attribution, matchback, or model output may describe credit without proving lift. |
| A report or deck is useful but produced by an interested party. | The producer role, sponsor role, outside checks, and limits placed near the claim. | Interest does not make evidence useless, but it raises the standard for method visibility. |
| A case, quote, or example is carrying the broader interpretation. | The population or decision to which the example is being generalized. | One vivid example can illustrate a risk without proving frequency, trend, or cause. |
Run the review in three passes
Pass 1. Name the claim.Rewrite the sentence without promotional, defensive, or headline language. If the wording says caused, lifted, proved, reduced, prevented, or transformed, mark it as causal until the method shows otherwise.
Pass 2. Test the support.Look for the closest source, the denominator, the comparison class, the source role, and the uncertainty statement. If one field is missing, record the specific document, table, method note, or sensitivity check that would reduce doubt.
Pass 3. Bound the next action.Decide what can be repeated now, what needs a caveat, and what must wait for more evidence. The final output should be a sentence a reviewer can defend, not just a color score.
How to use the result
A green score does not mean the claim is permanently settled. It means the current wording is proportional to the current evidence. A yellow score means the claim may still be useful if the missing piece is named. A red score means the claim should be rewritten, delayed, or treated as a lead for further checking.
Follow-up request map
The review should produce a concrete next request, not a vague concern. Use the weakest row to decide whether the team needs a source document, a denominator, a comparison, a method note, or narrower language.
| If the weak field is | Ask for | Hold back this language until then | Next guide |
|---|---|---|---|
| Source proximity | The original record, full report, data table, transcript, method appendix, or source page behind the summary. | Settled, confirmed, proven, industry-wide, or consensus language. | Source triangulation checklist |
| Denominator or base rate | The eligible universe, starting level, time window, excluded groups, and whether the mix changed. | Large, small, rare, common, growing, shrinking, or record-setting language. | Denominator framing examples |
| Comparison class | The fair baseline, peer group, holdout, matched market, prior period, or expected value. | Better, worse, overperforming, underperforming, or category-leading language. | Campaign baseline comparison checklist |
| Measurement design | The assignment rule, control definition, leakage check, outcome maturity, confidence interval, and sensitivity test. | Drove, caused, lifted, incremental, persuasive, or budget-proof language. | Measurement method selector |
| Claim wording | A revised sentence that names the evidence type, scope, caveat, and decision limit. | Any wording that is stronger than the source, denominator, comparison, or method can support. | Evidence-to-claim language matrix |
Meeting closeout
End the review by assigning the next evidence owner and writing the allowed claim in plain language. If the team cannot name the missing field, the issue is not ready for a confidence score. If the missing field is named but not available, the page, report, or budget note can still move forward with a narrower description.