Media framing

Composition mix and average claim checklist

Published June 2, 2026. Status: evergreen source page.

An average can move even when nothing meaningful changed inside any subgroup. The result may be driven by a different mix of people, cases, markets, products, respondents, placements, or outcomes entering the calculation.

Use this checklist when a story, dashboard, campaign report, survey, public-data release, or research deck relies on an average, rate, share, index, or aggregate comparison. The goal is to separate real within-group movement from a mix change that changes the headline number.

Start with the aggregate's job

Before accepting the read, ask what the aggregate is supposed to prove. A blended number is useful only when its ingredients are stable enough for the comparison being made.

Aggregate claim	Mix question to ask	Careful first read
"Average performance improved."	Did the share of easy, high-intent, high-value, or low-friction cases increase?	The blended average improved; segment-level movement determines whether performance improved broadly.
"The rate fell."	Did the population, reporting base, geography, age band, product mix, or eligibility rule change?	The aggregate rate fell under this mix; stable subgroups may tell a different story.
"The campaign produced better leads."	Did the campaign receive a different source mix, landing-page mix, form mix, or sales-follow-up mix?	Lead quality improved in the observed blend; source and destination mix must be separated before crediting the campaign.
"The survey shows a shift."	Did respondent composition, weighting, mode, field dates, or subgroup base change?	The reported share moved for this respondent mix; subgroup movement and weighting decide the strength of the claim.
"One market or group outperformed."	Were group sizes, baseline risk, opportunity, price, inventory, or exposure comparable?	The group led the aggregate result; the comparison needs equivalent opportunity and subgroup checks.
"The portfolio got more efficient."	Did spend move toward cheaper inventory, warmer audiences, smaller cells, or lower-value outcomes?	Efficiency improved for the blended portfolio; outcome quality and segment weights decide whether value improved.

Where composition shift hides

Audience and respondent mix

Different ages, regions, income bands, loyalty levels, survey modes, or reachable users can change an average even when every stable subgroup is flat.

Case and record mix

New intake rules, reporting channels, inspection coverage, backlog processing, or eligibility filters can change which cases appear in the record.

Campaign and placement mix

Creative rotation, device mix, inventory quality, source channel, geography, frequency, and landing-page mix can shift response metrics before impact is measured.

Outcome and value mix

Average order value, lead quality, pipeline value, visit quality, or matchback yield can improve because lower-quality outcomes disappeared or higher-value outcomes became easier to observe.

Minimum source packet

A mix-sensitive claim needs more than the final number. Ask for the fields that let a reader rebuild the aggregate and compare stable slices.

Segment weights

The share of observations, respondents, records, markets, placements, products, or outcomes in each group before and after the claimed movement.

Within-segment values

The average, rate, share, conversion, quality score, or outcome value inside each stable group, not only the blended total.

Eligibility and inclusion rules

Who or what could enter the calculation, what was excluded, whether rules changed, and whether records are preliminary or backfilled.

Stable comparison period

A window that is comparable on seasonality, product mix, reporting process, respondent source, sales coverage, and campaign setup.

Decision threshold

The size and direction of subgroup movement that would matter for the actual editorial, budget, or operational decision.

Composition-mix QA checklist

Question	Raise confidence when	Lower confidence when
Can the aggregate be rebuilt?	The source shows subgroup values and subgroup weights for each period or group.	The report shows only the final average, rate, share, or index.
Did the subgroup mix change?	Group shares are stable or the analysis separates mix effect from within-group movement.	A new population, product, market, source, respondent mode, or placement type entered the calculation.
Did each subgroup move the same way?	The direction holds across meaningful groups, or exceptions are explained.	The aggregate improves while key subgroups are flat or worse.
Is the denominator stable?	The eligible population, record base, survey base, or campaign universe is comparable.	The denominator changed because of reporting, eligibility, reachability, or data matching.
Are high-volume groups masking low-volume groups?	Large groups and small but material groups are both shown with minimum cell context.	A high-volume segment dominates the blended result while vulnerable, valuable, or decision-relevant groups disappear.
Does the decision depend on value, not just volume?	The report separates low-value and high-value outcomes, leads, visits, conversions, or records.	The same average treats all outcomes as equal even when value differs.
Was the cut chosen before results were visible?	Segments, weights, thresholds, and comparison periods were defined before the readout.	The report searches for the segment split that makes the aggregate story look strongest.

Rewrite the claim

Weak wording	Cleaner wording	Evidence needed to go stronger
"Average quality improved."	"The blended quality score improved; subgroup scores and weights determine whether quality improved broadly."	Stable subgroup movement, unchanged inclusion rules, and value-weighted checks.
"The public shifted its view."	"The survey share changed for this respondent mix, field period, and weighting approach."	Comparable sample source, subgroup bases, question wording, weighting, and uncertainty.
"The new strategy raised conversion rate."	"The observed conversion rate rose under this traffic, placement, and audience mix."	Protected comparison, stable traffic mix, preselected outcome, and leakage checks.
"The issue got worse for everyone."	"The aggregate rate rose; stable population groups need to be checked before generalizing."	Group-level rates, stable denominators, record completeness, and comparable time windows.
"The campaign found better customers."	"Reported customer value rose in the measured blend; source mix and qualification rules may explain the movement."	Source-level value, follow-up coverage, deduplication, and a comparison group.

Simple decomposition pass

You do not need a complex model to catch the most common mix errors. A simple two-pass comparison often changes the story.

1. Hold the mix constant.

Apply the old subgroup weights to the new subgroup values. If the aggregate result weakens, part of the story was a mix change.

2. Hold subgroup values constant.

Apply the new subgroup weights to the old subgroup values. If the aggregate moves, the mix alone could explain part of the result.

3. Inspect the remaining movement.

The stronger claim belongs only to the portion that remains after mix changes, denominator changes, and data-rule changes are visible.

Pair with

For media claims, pair this checklist with the denominator framing examples, public records and denominator checklist, survey and poll claim checklist, before-and-after trend checklist, and claim confidence rubric.

For campaign readouts, pair it with the campaign readout QA checklist, campaign KPI dictionary, outcome quality scorecard, audience selection bias checklist, and attribution window and conversion lag checklist.

Takeaway

Averages are not neutral summaries when the ingredients change. Before trusting the aggregate, ask whether the people, records, markets, placements, or outcomes inside it are the same enough for the comparison to carry the claim.