How to Prepare Questions for Evaluating Public Statistics

Public statistics can help community groups understand needs, assess programs, and explain human-rights concerns. To use them responsibly, prepare questions that clarify what was counted, who may be missing, and what conclusions the figures can support.

1. Start with a decision or learning goal

Before searching for a number, say what the group needs to decide or understand. A statistic is useful only in relation to a question. “How many people are experiencing housing insecurity?” may be a starting point, but it does not yet explain whether you need a local service estimate, a change over time, or a comparison between groups.

Write a one-sentence purpose. For example: “We want to understand whether residents in the east district face longer waits for a public service, so we can decide what to investigate with community partners.” This keeps the research focused and avoids collecting figures simply because they are available.

Then identify the intended use:

• Learning: What do volunteers or participants need to understand?
• Planning: What information could shape a project, outreach schedule, or service location?
• Accountability: What claim needs checking before a public meeting or report?
• Advocacy: What evidence can support a concern, and what additional evidence is needed?

Separate a factual question from a proposed explanation. “Did reported wait times increase?” is factual. “Did a policy change cause the increase?” asks about cause and needs stronger evidence. Public data may show a pattern without explaining why it occurred.

2. Turn the topic into answerable questions

Draft questions in plain language and make each one specific enough to investigate. Define the population, place, time period, and measure. If these are not clear, different people may answer different questions while using the same words.

A useful question often identifies:

• Who or what: residents, service requests, households, or organizations.
• Where: a neighborhood, municipality, region, or country.
• When: a named year, quarter, or consistent period.
• What is measured: people served, applications received, reported incidents, or another defined quantity.
• How it will be compared: across time, locations, or groups, if comparison is appropriate.

For example, replace “Are older residents being left out?” with “For the latest year available, what share of residents aged 65 and older in the district used the named program, and how does that compare with the district’s estimated population in that age group?” This version exposes a denominator question: the count served alone cannot show a participation rate.

Prepare a short set of questions in layers:

1. Availability: Is there a public source that measures the issue we care about?
2. Definition: How does the source define the population and measure?
3. Coverage: Which people, events, places, and periods are included or excluded?
4. Comparison: Are the figures comparable across groups or time?
5. Use: What can we responsibly say, and what remains uncertain?

Keep the list manageable. A volunteer team can usually begin with one main question and a few supporting questions, then refine them as it learns what the data can answer.

3. Find out how the figures were produced

A published table is the end of a process. The original source may be a census, administrative records, a survey, a registration system, or a compilation from several organizations. Each method captures a different slice of reality.

Ask the source or consult its accompanying notes:

• Who collected or supplied the information, and for what purpose?
• Was the information collected from everyone in scope, a sample, or only people who contacted a service?
• What are the unit and denominator: people, households, visits, reports, or eligible cases?
• How are categories defined, and have those definitions changed?
• What is the collection period, publication date, and update schedule?
• Were records checked, revised, estimated, or suppressed for privacy?

Administrative data can be detailed, but it represents interactions with an institution rather than everyone who may have a need. A count of people who filed complaints, for instance, does not establish the total number who experienced the underlying problem. A survey can reach beyond service users, but its estimates depend on who responded and how questions were asked.

Record source details as you work. Save the publisher, table or dataset name, link, access date, time period, definitions, and any methodological notes. If the group later shares a figure, these details help readers trace it and prevent a number from becoming detached from its context.

4. Check coverage, missingness, and rights-related risks

Human-rights literacy includes asking whose experiences are represented and whose are obscured. A category may be absent because it was not collected, because respondents could not safely disclose it, or because a small count was withheld. Missing data is not proof that a group is unaffected.

Ask whether the figures can be examined by relevant characteristics, such as age, location, disability, or another locally meaningful category. Do not assume every source collects these details or that every comparison is appropriate. Consider whether a group is defined consistently and whether the data collection could expose people to stigma, retaliation, or unwanted identification.

Review these points before drawing a conclusion:

• Are people without stable housing, formal documentation, internet access, or contact with institutions likely to be undercounted?
• Are language, accessibility, or trust barriers likely to affect participation or reporting?
• Does “not recorded” mean unknown, not asked, or deliberately withheld?
• Could combining small groups reveal an individual or a sensitive circumstance?
• Do categories reflect the community’s terms and realities, or an administrative convenience?

Use only the detail needed for the question. When public tables suppress small counts, do not try to reverse-engineer identities from other information. If a group’s experience is missing, report that limitation and consider safe, voluntary ways to learn more through established community relationships. Do not collect sensitive personal accounts casually or promise confidentiality the project cannot provide.

5. Assess comparisons and uncertainty

Comparisons can reveal patterns, but only when the underlying figures are sufficiently alike. A city and a rural area may use different reporting systems. A program may change eligibility rules. A survey may change its wording or sampling approach. In each case, a difference in published figures might reflect measurement changes rather than a real shift in conditions.

Use this compact review before comparing:

Check	Question to ask	Why it matters
Definition	Did the measure and categories stay the same?	A changed definition can create an apparent trend.
Coverage	Do the sources count comparable people and events?	Different coverage can make a comparison misleading.
Denominator	Are counts adjusted for population or opportunity?	Larger groups often produce larger raw counts.
Time	Do the periods and seasonal patterns align?	A partial year or seasonal cycle can distort the result.
Uncertainty	Are estimates or small samples involved?	Small differences may not be meaningful.

For rates, check how they were calculated and what denominator was used. A rate per population may be more informative than a raw count when comparing places of different sizes, but it still does not resolve differences in reporting, eligibility, or access. Ask whether the source provides a margin of error, confidence interval, or other uncertainty information. If it does, explain it in ordinary language; if not, do not imply that a precise-looking number is exact.

When a source reports estimates, round appropriately and preserve its caveats. Avoid ranking small areas or groups on tiny differences unless the source supports that use. If the comparison is uncertain, describe the observed pattern as tentative and identify what would help clarify it.

6. Organize a community review

A volunteer organization does not need a specialist for every first-pass review. It does need a clear process that gives people time to question assumptions and keeps the source trail visible. Assign practical roles: one person finds and records sources, another checks definitions and calculations, and a facilitator makes sure the group distinguishes evidence from interpretation.

For a community project meeting:

1. Share the decision question and the source link in advance.
2. Ask participants what they think the measure captures and what it may miss.
3. Read the source’s definitions and notes together before discussing implications.
4. Record disagreements, unknowns, and alternative explanations without forcing consensus.
5. Agree on the narrowest claim the evidence supports and what further information is needed.

If participants have direct experience relevant to the issue, treat it as knowledge that can shape follow-up questions. It is not a substitute for a population estimate, and a public statistic is not a substitute for lived experience. Keep participation voluntary, explain how notes will be used, and avoid recording identifiable details unless there is a clear, safe reason and appropriate agreement.

For a small organization with limited time, use a one-page worksheet: purpose, main question, source, definitions, coverage, limitations, supported claim, and next step. This simple record is easier to revisit than a long spreadsheet without context.

7. Troubleshoot common problems

The source has a number but no clear definition. Search the publisher’s methodology notes, glossary, or data dictionary. If the definition remains unclear, contact the publisher through its public inquiry channel or choose a better-documented source. Mark the figure as provisional rather than silently guessing.

The number is missing for a group or neighborhood. Check whether the value is suppressed, not collected, unavailable, or reported under a different boundary. Do not treat an empty cell as zero. State exactly what the source does and does not provide, then look for an independent source with a compatible definition.

Two credible sources disagree. Compare collection dates, populations, definitions, geographic boundaries, and methods. They may be answering different questions. Present both with their scope if neither can be resolved; do not average them automatically.

A trend begins or ends abruptly. Check for a change in data collection, service access, reporting practices, or boundary definitions. Look for source notes and compare overlapping periods if available. A sharp change may be real, but the chart alone cannot establish its cause.

The group wants a causal answer. A before-and-after chart or geographic difference is not enough by itself to show that a program or policy caused an outcome. Note plausible alternative explanations and seek an evaluation design or additional evidence suited to the causal question.

8. Share findings with proportionate claims

When writing a project brief, meeting note, or public explainer, state the source and time period near the figure. Describe the population and measure in plain language, and include a limitation that could affect interpretation. Keep observed results separate from explanations and recommendations.

A useful reporting pattern is: “The [source] recorded [measure] for [population or area] during [period]. The source defines this as [brief definition]. It does not include or establish [key limitation], so the figure supports [careful claim] but cannot show [unsupported claim].”

Use direct, respectful language about people and groups. Avoid labels that imply blame, and do not present a group’s absence from a dataset as absence of need. If the source changes or a better estimate appears, update the project’s notes and correct any material public claim. The goal is a question set that helps the community learn accurately, decide what to investigate next, and communicate what the evidence can support.