Features
Part of Magazine copy editing: a practical guide to clean, accurate copy
Case study: fact-checking a fictional magazine data claim
A fictional magazine catches a survey claim that used the wrong denominator, then rewrites the headline, chart and caption to match what the data supports.
What to take away
- A percentage is only as good as the denominator underneath it.
- Two surveys with different questions and reference periods cannot be subtracted from each other.
- An eight-point move from 20 percent to 28 percent is a 40 percent relative rise, never a "40-point" one.
- A lead smaller than the margin of error is too close to call, not a finding.
- The correction trail is worth keeping even when nothing reaches print.
This case is invented. The magazine, the survey, the people and every figure below are fictional. It walks through the checks a desk would run, not a real study.
The submitted claim
Local Measure receives a feature on evening bus use. The draft says late-shift ridership rose 40 percent in a year, proving a pilot route transformed commuting. Its chart shows 20 percent for 2025 and 28 percent for 2026. The headline calls that a 40-point leap.
Four claims to check
- Who was measured
- How much changed
- Over what period
- Whether the route caused it
The checker splits the sentence into four claims: who was measured, how much changed, over what period, and whether the route caused it. The source packet holds two survey summaries, a transit-agency press release, a dashboard export and an interview with the pilot manager.
Step 1: recover the original data
The checker asks for questionnaires, methods, full tables, revision dates and definitions. The 2025 survey asked whether anyone in the household used any bus after 8 p.m. in the past month. The 2026 survey asked whether the respondent used the new route after 7 p.m. in the past week.
Two surveys, two questions
2025 survey
- Question
- Any bus after 8 p.m.
- Population
- Household member
- Window
- Past month
- Comparable?
- No
2026 survey
- Question
- New route after 7 p.m.
- Population
- Respondent
- Window
- Past week
- Comparable?
- No
Different questions, different populations, different windows. The two percentages do not measure the same behavior, so subtracting them is meaningless.
The International Fact-Checking Network's Code of Principles asks for transparent source selection, the best available primary evidence, enough method detail for readers, and a published corrections policy. On that basis the team writes down why the dashboard alone cannot carry the comparison.
Step 2: correct the arithmetic language
Suppose the measures were comparable. Moving from 20 percent to 28 percent is eight percentage points, which is a 40 percent relative increase. The headline's "40-point" phrase is wrong on both readings. The checker records both calculations on the claim sheet and flags the chart label for replacement.
Points versus percent
- 20%2025 estimate
- 28%2026 estimate
- 8percentage points
- 40%relative increase
Step 3: test uncertainty and causation
The tables call their figures sample estimates and print margins of error beside them. They do not pin down a precise change across all riders. The pilot also started the same month as a fare discount and two large employers redrawing shift schedules. Nothing in the packet isolates the route.
The Journalist's Resource tip sheet on the margin of error makes the point plainly: a sample never mirrors the population exactly, and the margin shrinks only as the sample grows. A two-point lead inside a three-point margin is too close to tell.
Step 4: inspect the whole package
A bus-stop caption dates the photograph to opening day. The image metadata places it five days later. So the caption becomes "during the pilot's first week."
A pull quote from the manager is accurate as far as it goes. But the layout drops the clause limiting it to surveyed riders. The full phrase goes back in.
The final wording
The published feature draws no trend from the two surveys. It reports what each survey found among its own defined sample, states that the questions and periods differ, and presents agency boarding records as a separate measure. It describes the pilot, the discount and the shift changes without assigning cause.
The headline becomes "What a New Evening Bus Survey Can and Cannot Show." The chart prints each question verbatim beside its estimate and its uncertainty. A source note explains the noncomparable periods.
Decision record
| Item | Initial wording | Final decision |
|---|---|---|
| Trend | 40 percent rise | Removed as noncomparable |
| Headline | 40-point leap | Replaced |
| Cause | Route transformed commuting | Narrowed to observed measures |
| Caption | Opening day | First week |
| Chart | Bare percentages | Questions, periods, estimates, uncertainty |
Common questions
Why not average the two surveys?
Their questions, reference periods and samples are not established as comparable. Averaging incomparable numbers produces a figure that describes neither survey. The desk would need a single instrument run twice before any pooled number means anything.
Is 20 percent to 28 percent a 40 percent increase?
Arithmetically, yes, as a relative increase. It is eight percentage points, and only meaningful if the two measures are comparable and precise enough to separate. The draft's "40-point" phrasing was wrong under either reading.
Does an agency press release count as primary evidence?
It is primary evidence of what the agency said, not of the underlying result. Ask for the boarding counts, the method note and the revision date behind it. Where the release and the data diverge, the data wins and the divergence goes in the claim sheet.
Was a public correction required?
No. Every error was caught before release, so there was nothing published to correct. The internal decision record was kept anyway, because it shows the next checker which traps this beat sets.
