Checking Statistical Assumptions in Practice: What to Plot, What to Trust, What to Do Next (2026)

Quick answer: Statistical assumptions are not a paperwork ritual. Each one names a specific condition under which the method’s error rate and interval coverage are trustworthy — and the checks that matter most are design reasoning and targeted plots, not mechanical test scripts. Three checks carry almost all the weight for ordinary models: independence (usually a design question, not a data question), equal-variance behavior across fitted values, and normality of the residuals around the model — never of the raw outcome.

Part 10 of a practical statistics series for researchers. If the basics are new, start with p-values, effect sizes, and confidence intervals.

Assumptions are claims about the model, not about the data

An assumption statement such as adds that the errors behave like symmetric random draws around the fitted line. That is a statement about the process that generated the deviations — errors of the model — not a starting promise about the raw column of numbers you logged.

This distinction is the source of the most widespread assumption error in applied work across whole fields: a paper logs a skewed biomarker, runs a normality check on the raw outcome, reports that normality fails, and then swaps in a rank test. The check has tested a claim the analysis never made. In regression, the relevant question is whether the deviations from the fitted surface are reasonably symmetric — a different question, answerable only once the model exists (a systematic review by Ernst and Albers documents this pattern extensively).

Practical implication: assume-checking starts after a candidate model is fitted, not before, and the unit of examination is the residual, not the raw outcome.

The three families of assumptions

For the ordinary linear-style methods that dominate applied work, there are three families of assumptions, and they differ fundamentally in what suspicion should befall them.

• Independence, meaning the errors behave unconnectedly, with no shared hidden cause and no systematic ordering. It is usually broken or preserved at design time. A data-driven diagnostic can narrow suspicion, but it cannot manufacture independence where the design never had it.
• Equal-variance behavior (homoscedasticity), meaning the spread of the residuals stays roughly the same across the fitted values and across groups. It is a data-and-model question, first answered by a plot of residual variance across fits rather than by a single-number test.
• Shape conditions (normality of errors, less commonly linearity of the mean relation), meaning the deviations behave like symmetric symmetric draws around the fit and the mean relation is honest. The most commonly inspected — and the least dangerous to get slightly wrong for inference in many realistic settings, compared with what reviewers typically fear.

This ordering matters because researchers frequently rank assumptions by their check-script presence rather than by the severity of their failure. Designs that break independence are the ones that need rework — clustering, pairing, time-orders — yet they are the least diagnosed; whereas shapes get the most p-value ink and the least attention from plots.

Checking the residuals, not the raw outcome

Once a model is fitted, you work with residuals — actual variation minus fitted variation. Then:

• Plot residuals against fitted values. A healthy picture shows roughly symmetric scatter with similar-width bands along the fitted axis.
• Plot residuals against each well-represented predictor. A healthy picture shows no systematic crescents or wedges.
• Forecast the normality behavior with a residual Q–Q plot. Points scatter around a straight band when the error structure is roughly Gaussian. Curved or S-shaped patterns signal issues to interpret together with sample size and domain knowledge.

Uniform error behavior — such as bounded scales, count variance tied to the mean, or heavy-tail-signed errors — may show up as wedges or curvature in the residuals. Those are visible signals to revisit the model family (see the regression choice guide), not to delete the subjects who produced them.

What the formal tests can and cannot tell you

Every formal normality check, such as the common Shapiro–Wilk one, is a null-hypothesis test. It answers one narrow question and cannot answer the practical one.

• What a small p-value does establish: evidence against the exact null, at that sample’s noise level. A smaller sample shrinks the detection power toward zero; a larger one magnifies even scientifically trivial departures toward certainty.
• What it does not establish: how consequential the departure is, or what an applied researcher should do next. Tests of this kind offer no severity-of-consequence report, and a diagnosis from p-values alone routinely leads away from needed model-reaching.

A non-significant result is not a pass, and a significant one is not automatically a fail. Because approximate methods are robust to moderate non-normality in central regions with reasonable sample sizes, the common two-step script of test-then-switch has no statistical archetype behind it (as the rank-tests article elaborates for the switch itself). Diagnostics should weigh severity via plots against the available sample size and the intended analysis; mechanical checking scripts cannot.

Homoscedasticity: plot, then respond

A frequent funnel-shaped residual-versus-fitted plot is the signature of variance that grows with the level of the outcome. It is common across logs of positive quantities, ratings with floor and ceiling behavior, and anything fit for multiplicative character.

Useful responses, roughly in honesty order:

• Model the variance as it actually behaves. When the spread pattern is substantive and structural, heteroscedasticity-aware variance or robust-standard-error methods give honest standard errors without pretending the condition holds.
• Transform the outcome only when that matches the science. Logging waiting times or concentrations when the scale is genuinely multiplicative can legitimately produce a well-behaved model — and restores interpretable coefficients. A transformation performed purely to silence a warning, with no scientific reading, is much harder to defend downstream as explained in the regression guide.
• Keep the estimand clear. If the claim is about a mean on the original scale, note that a variance-based response changes what you quote, or reintroduce the claim honestly at the fitted scale.

Do not tabulate separate variance checks by group and then discard a whole design because one group was thinly sampled; a noisier group estimate is not evidence that equal spread is broken overall.

Independence: the assumption reviewers rarely catch

Independence is usually a design and data-collection question. Repeated measurements per subject, clustered households, site-ordered measurements, and matched sites do not have it — no matter how clean the residual plot appears.

What this means practically:

• The residual plot is not a test of independence. Residual patterns from clustered data can look entirely innocent while the shared-cause structure quietly inflates the effective sample size.
• The design should drive the analysis family. Repeated measures and multilevel structures call for the corresponding model choices (see the mixed-models article).
• When independence is broken without any model in hand to reflect it, quote the analysis honestly as an approximation and discuss the direction of the doubt it introduces.

The habit worth building is to ask, before any fitting, what observations could share causes — because the modeling decisions are easier to make at design time than to reverse-engineer after.

What to do when a check actually fails

Seasoned practice is a chain of questions, not a single panic move:

• First look for explanations in the design and data. One wildly influential record; a batch effect; an unmodeled grouping; an entry error. Fixing these is model repair, not assumption failure.
• Then consider whether the model family is wrong rather than the data. Bounded scales, multiplicative noise, nonlinear mean relations, and zero-heavy counts suggest a different model family rather than a heroic residual rescue.
• Only then consider transformation or robust alternatives — and quote what the change does to the estimand you originally declared.
• Finally, re-diagnose. After any change, the residual plots and design-reasoning revisit the question with the modified model in hand.

Failing assumptions is a signal to think, not a requirement to delete observations or mechanically insert a non-parametric swap. The estimation story in the missing-data article illustrates the related truth for missingness: an assumption has a direction of bias attached, and that direction needs discussion rather than silent handling.

Reporting assumption checks

A reproducible analysis report should include:

• the design reasoning for independence, stated in plain words;
• the residual plots actually inspected (residual-versus-fitted and Q–Q);
• what any formal test contributed, with its sample-size context;
• what was changed as a result and how the estimand was affected;
• any remaining assumption risk, named explicitly rather than suggested to be absent.

The reproducible-workflow article covers the machinery that makes these artifacts durable; here the only point is that assumption reasoning is recorded, not merely performed.

Common mistakes

• Testing normality on the raw outcome instead of the model residuals.
• Treating a non-significant normality test as a claim that everything else is fine.
• Switching to a rank test on the basis of a mechanical check script rather than the estimand.
• Ignoring clustered, repeated, or matched designs because residuals happened to look tidy.
• Reading homoscedasticity from group-level variance tests alone.
• Deleting influential observations without recording why and how the result changed.
• Transforming outcomes without stating what the coefficients now mean.
• Treating passed checks as proof that no assumption risk remains.

Key takeaways

• Assumptions describe the model’s error process, not the raw data; residual-centered checking is the honest starting point.
• Visual diagnostics of residuals outrank formal tests for severity assessment, with sample-size context always attached.
• Independence is primarily a design property — checking it after modeling cannot recover it.
• Homoscedasticity questions are best answered by residual-versus-fitted plots and responses that preserve an honest estimand.
• Failures call for a question chain: repair, respecify, transform or go robust — in that order, with re-diagnosis after any change.
• Assumption reasoning belongs in the report, not in private memory.

Challenge

Take a model you actually fit recently, and do a five-minute assumption review: state the design reasoning for independence in one sentence; look at the residual-versus-fitted plot and describe two things it shows; state what the normality situation of the residuals means for the claim you made; and name what you would have quoted differently had the design been clustered. If any of those four steps was skipped originally, note whether the reported conclusion is safe or needs a caveat.

Running assumption checks on real fitted models? Ampersand Academy offers one-to-one mentoring on statistics and data analysis — part of Mahadhi, a development and digital marketing company.

—

FAQ

Large samples make formal normality flags fire at trivial departures, but they do not remove design failures. Independence and homoscedasticity matter at every sample size, and residual plots stay worth reading.

• Is it safe to skip assumption checks when the sample is large?

Of the residuals for a regression model, and of the pooled error space for group comparisons under a shared-shape argument. Raw-outcome normality of the measured variable is a different and usually irrelevant question.

• Do I check normality of each group or of the residuals?

No. It reports evidence against exactly-normal errors at that sample size, with no severity attached. Judge severity from plots and context, and choose around the estimand rather than the flag.

• Is a significant Shapiro–Wilk result a command to switch tests?

Read the spread of residuals along the fitted values: a uniform band is the healthy shape; funnel or wedge patterns signal variance that varies with level. Group-level variance tests alone are a weak substitute.

• What is the fastest way to judge homoscedasticity?

Usually not. Clustered or ordered errors can leave no visible structure. Independence is best assessed from the design: repeated measurements, clusters, matched sites, and time ordering are the usual suspects.

• Can residual plots detect broken independence?

Not as a routine. Occasional records with a documented cause can be addressed transparently, but routine trimming to tidy diagnostics hides structure and biases the estimate in ways you will not be able to quantify.

• If assumptions fail, is deletion of outliers the standard remedy?

Enough to reconstruct the reasoning: which diagnostics were read, what they showed, what changed as a result, and what residual risk remains. P-values alone tell the reader almost nothing worth acting on.

• How much detail belongs in the report about assumptions?