Math · Problem-Solving and Data Analysis

SAT Statistical Claims: The Complete Guide

A numerical difference is only part of a statistical argument. Learn to read the study design, identify who the result can describe, and decide whether it supports association or a causal conclusion.

Section: Math
Domain: Problem-Solving and Data Analysis
Calculator: Available throughout Math
Focus: Interpret first; calculate second

Ask two different questions about a study

Statistical-claims questions ask whether a conclusion follows from how a study was conducted. One question is about scope: which population can the result reasonably describe? The other is about cause: can the treatment be credited with producing a difference? A study can answer one well without answering the other. Evaluate them separately.

An observational study records existing choices, conditions, or outcomes. Researchers might compare people who already follow different routines. An experiment deliberately imposes treatments. A randomized experiment assigns participants to treatments by chance. The researcher doing something is not enough; the random assignment is the key design feature for a causal comparison.

A survey can be observational even when its sample was selected randomly. Random sampling concerns who enters the study. Random assignment concerns which treatment each participant receives after entering. The word random by itself is incomplete: identify the action that is random. This distinction is the foundation for nearly every question in this skill.

Mark selection and assignment separately

First, find the sentence describing recruitment or selection. Were people randomly selected from a named population, invited as volunteers, or taken from a convenient location? That determines the strength and scope of population inference. A random sample of one school's students represents that school's sampling population, not automatically all students everywhere.

Second, find the sentence describing treatments. Were participants assigned by chance, assigned according to preference, or simply observed in groups that already existed? Chance assignment helps balance preexisting characteristics across treatment groups, reducing confounding. Groups formed by preference can differ in motivation, resources, or other factors related to the outcome.

Third, read the conclusion's strength. Associated with describes a relationship. Caused, led to, and resulted in make a causal claim. All residents makes a broader scope claim than participants in this study. Match both the causal wording and the population wording to the design, rather than choosing an answer merely because its numerical comparison is correct.

The design determines the conclusion

Selection	Treatment assignment	Typical supported scope
Random sample from a defined population	None; observe existing groups	Population association; no causal attribution from the design alone
Volunteers or convenience sample	Random assignment	Causal evidence for the studied participants; broader generalization is limited
Random sample from a defined population	Random assignment	Causal comparison with support for inference to the sampled population
Volunteers or convenience sample	None or self-selected treatment	Descriptive association among those studied; both broader inference and causation are limited

A confounding variable is related to both the group or treatment status and the outcome, making their effects hard to separate. If students who choose a tutoring program score higher, motivation may influence both program enrollment and scores. The observed difference could reflect tutoring, motivation, prior preparation, or several influences together.

Random assignment does not guarantee that groups are identical on every characteristic. It makes systematic allocation by those characteristics less likely and supports a fair comparison through chance balancing. Especially with small groups, some imbalance can remain. A conclusion should reflect evidence about a treatment effect, not claim that randomness eliminates all uncertainty.

A comparison group gives the treatment result a reference. A placebo can help separate a treatment's specific effect from expectations, and blinding can reduce behavior or measurement differences caused by knowing group assignments. These are useful protections, but none turns a convenience sample into a representative random sample of a broader population.

Audit a claim against the study description

Name the population the claim discusses and the group from which participants were actually selected.
Identify random sampling, voluntary response, or convenience selection. Do not infer an unstated random process.
Decide whether researchers imposed a treatment and whether assignment to treatments was random.
Identify plausible confounding or design problems, including unequal conditions or substantial dropout.
Choose wording that matches the design's population scope and causal strength, without treating evidence as certainty.

The response variable should match the conclusion. If a study measures short-term recall, it does not directly establish long-term course performance. A question can offer a conclusion with appropriate population scope but the wrong outcome. Read what was measured, when it was measured, and which conditions were compared.

Distinguish missing information from a demonstrated flaw. If the prompt does not state that selection was random, do not assume it was. But do not invent a particular hidden bias as though it were known to exist. It is enough to say that broader representativeness is not established or that an observational design leaves alternative explanations.

Original worked examples: easy to hard

Worked example

Easy: random selection does not create causation

Researchers randomly select 500 adults from a city's resident list. They record each adult's usual walking time and a wellness score. Adults who walk more tend to have higher scores. Which conclusion is supported by this design?

AWalking more necessarily causes a higher wellness score.
BThere is evidence of an association between walking time and wellness among the city's adults.
CThe relationship must hold in every city.
DRandom selection proves that no other variable affects wellness.

Show the solution

The adults were randomly sampled from the city, supporting inference to that sampled population under ordinary survey assumptions.
Walking habits were observed rather than randomly assigned. Other differences could affect both walking and wellness.
Choice B describes an association with the appropriate city scope. Choice A incorrectly turns random sampling into random treatment assignment, while C extends the population beyond the sampling frame.

Answer: B

Worked example

Medium: volunteers in a randomized experiment

Eighty volunteers at one community center are randomly assigned to practice a memory task with either background music or silence. The two groups follow the same schedule and take the same final task. The music group performs better, and the analysis provides strong evidence of a treatment effect. Which conclusion is most justified?

AMusic improves memory-task performance for every adult.
BThe volunteers are a random sample of all community residents.
CThe experiment supports a causal effect of the music condition for the studied participants, with limited basis for broader generalization.
DThe study cannot support any causal conclusion because participants volunteered.

Show the solution

Volunteering limits population generalization. The volunteers may differ from other residents.
Random assignment, comparable conditions, and the stated evidence support attributing the difference to the tested condition for these participants.
Choice C keeps the causal conclusion and the scope separate. Choice D incorrectly treats nonrandom recruitment as though it prevented random assignment from supporting a causal comparison.

Answer: C

Worked example

Hard: identify a fair redesign

Original nonrandom comparison of seedling treatments
Condition	Source of seedlings	Light exposure
Nutrient solution	Nursery A	Six hours per day
Plain water	Nursery B	Three hours per day

A researcher wants to isolate the causal effect of the nutrient solution on growth. Which redesign best addresses the original study's confounding?

AUse more seedlings from the same two nurseries while preserving the treatment and light differences.
BCompare only the tallest seedling in each original group.
CAsk nursery owners which treatment they expect to work better.
DRandomly assign seedlings from a common pool to solution or water and keep light and other growing conditions comparable.

Show the solution

In the original comparison, treatment varies together with nursery source and light exposure. Either could help explain a growth difference.
Random assignment from a common pool reduces systematic preexisting differences. Comparable light and growing conditions prevent the treatment contrast from being bundled with a light contrast.
Choice D is the best redesign. Choice A increases sample size without removing the systematic differences, so it does not isolate the solution's effect.
The redesign concerns a causal comparison for the seedlings studied. It does not automatically establish the same effect for every plant species or growing environment.

Answer: D

Common traps in causal language and scope

A control group is useful, but its existence alone does not ensure a fair experiment. If researchers assign the healthiest plants to the treatment and weaker plants to the control, starting condition is mixed with treatment. Random assignment addresses allocation; keeping conditions comparable addresses additional differences introduced during the experiment.

Do not reject all information from a limited study. An observational study can describe an association, and a randomized experiment with volunteers can support a causal comparison for participants. The correct answer often makes a careful positive claim rather than saying nothing can be learned. Match the conclusion to what the design does support.

Hard questions: population boundaries and bundled changes

A sampling frame is the actual list or group from which the sample is drawn. Randomly selecting names from a city's registered library users can support inference about that listed population, subject to response and coverage assumptions. It does not automatically represent residents who never registered. The broad word city in the story does not erase the narrower frame.

A treatment can contain several simultaneous changes. If one randomized group receives new software and additional coaching while the other receives neither, the design can compare that combined package with the control. It cannot isolate the software's effect alone. An answer naming only one component can be too strong even though assignment was random.

Dropout can weaken a comparison when it differs systematically across groups. Suppose participants who find a treatment difficult leave before final measurement, while all control participants remain. Comparing only finishers may no longer preserve the initial random allocation's protection. Read any participation or follow-up details that the question explicitly provides.

If a study includes both random sampling and random assignment, separate the two steps in your explanation: sampling supports inference to the population sampled; assignment supports a causal treatment contrast. Avoid saying the result applies to all people or all settings. Even a strong design has boundaries defined by the population, treatment, outcome, and conditions actually studied.

Practice the two-decision method

Use a 20-minute session. For five minutes, classify brief study descriptions using two columns: selection and assignment. Include a random survey, a volunteer randomized trial, and a convenience observational comparison. Write the strongest supported population and the strongest supported causal wording for each. This builds the distinction before you face distracting answer choices.

Spend ten minutes on Solar statistical-claims questions. Underline the sampling frame, circle the assignment mechanism, and box the measured outcome. When you reject a choice, name its exact overreach: wrong population, unsupported causation, unmeasured outcome, or an assertion of certainty. Avoid the vague note bad study, which hides the decision you need to improve.

Who could enter the sample?
How were treatment groups formed?
What alternative explanation remains?
Does the conclusion match the measured outcome?
Does the claim stay within the studied population and conditions?

Finish by redesigning one flawed study in a sentence. Add random sampling if the goal is population representation; add random assignment and comparable conditions if the goal is a causal treatment comparison. Revisit these decisions alongside margin-of-error problems on another day. More precise estimates and stronger causal designs improve different aspects of evidence.

Keep reading — it’s free

Create a free Solar account to unlock the rest of this evaluating statistical claims: observational studies and experiments guide, plus adaptive practice on exactly this skill.

The design determines the conclusion
Audit a claim against the study description
Original worked examples: easy to hard
Common traps in causal language and scope
Hard questions: population boundaries and bundled changes

Unlock the full guide Already have an account? Log in

Evaluating statistical claims: observational studies and experiments: FAQ

What is the difference between an observational study and an experiment?

An observational study records existing conditions or choices. An experiment imposes treatments; a randomized experiment assigns participants to those treatments by chance.

Does random sampling prove causation?

No. Random sampling supports inference to the population sampled. Random assignment to treatments is the key design feature for a causal comparison.

Can an experiment with volunteers show a causal effect?

A properly conducted randomized experiment can support a causal comparison for its participants. Volunteer recruitment limits automatic generalization to a wider population.

Does a larger sample remove confounding?

No. More observations do not remove systematic differences between self-selected groups. Random assignment and comparable study conditions address the causal comparison.