Quick answer: Missing data is not just an annoyance to be deleted or filled in automatically. It is a set of assumptions about why the data are missing and what that means for your conclusions. If you ignore those assumptions, you can bias estimates, shrink variability in misleading ways, or pretend a result is more certain than it is. The first move is not a fancy imputation method. It is to make the missingness mechanism and your chosen handling strategy visible.
Part 6 of a practical statistics series for researchers. If the basics are new, start with p-values, effect sizes, and confidence intervals.
Table of Contents
Why missing data is a modeling problem
Every missing value is a gap with a history. The gap exists for a reason, and that reason matters.
A survey respondent may skip an income question. A sensor may fail. A participant may drop out of a study. A lab value may be missed because the visit was missed. Each case has a different flavor, and the appropriate handling depends on the likely process that produced the missingness.
That is why “just delete it” or “just impute the mean” are not real strategies. They are defaults that can work in narrow situations and fail quietly in others.
A better approach begins with three questions:
- How much is missing?
- Where is it missing?
- Why might it be missing?
Those questions shape the rest.
The three broad missingness ideas
Statisticians often talk about three broad mechanisms.
- Missing completely at random. The chance of missingness does not depend on observed or unobserved values. This is the cleanest case and also the least common in practice.
- Missing at random. The chance of missingness depends on observed data but not on the missing value itself, after accounting for what you observed. This is a workable assumption in many analyses if handled carefully.
- Missing not at random. The chance of missingness depends on the missing value itself, or on something you did not measure. This is the hardest case because the missingness carries information you cannot directly check from the observed data.
You do not always know which case you are in. That uncertainty itself is important.
What complete-case analysis does
Complete-case analysis uses only the rows with no missing values for the variables involved.
That sounds simple and safe. Sometimes it is acceptable. But it has downsides.
If the missingness is related to the outcome or predictors, complete-case analysis can change the effective sample in a biased way. You may end up analyzing a non-representative subset without realizing it.
Complete-case analysis can also waste information. If only a few values are missing here and there, the loss may be small. If missingness is widespread, the loss can be severe.
The key point is not that complete cases are always wrong. The key point is that the decision should be deliberate, not automatic.
Mean imputation is not a safe shortcut
Replacing a missing value with the mean, median, or mode is tempting because it is easy.
The problem is that it treats the missing value as if it were known with certainty, while in reality it is not. That can shrink variability, distort relationships, and make confidence intervals too narrow.
Mean imputation can also bias associations, especially when the missingness is related to other variables. It is not a neutral fill-in. It is an assumption-laden shortcut.
For quick summaries, simple imputation can sometimes be okay for very small amounts of missingness under strong assumptions. But as a default for analysis, it is usually too crude.
Why proper imputation matters
Better imputation methods try to preserve the structure of the data rather than replace a missing number with a single fixed value.
The goal is not to invent the “true” missing value. The goal is to fill in plausible values in a way that reflects uncertainty and keeps relationships among variables more intact than a naive fill-in.
Good imputation usually means:
- using other observed variables to predict or draw missing values,
- reflecting the uncertainty from not knowing the true value,
- and checking whether the imputed distribution looks reasonable.
Imputation is not magic. It cannot recover information that was never collected. But it can be much better than deletion or crude fill-ins when used carefully.
Multiple imputation is often the practical standard
Multiple imputation is a common approach because it takes uncertainty seriously.
Instead of filling each missing value once, it creates several completed datasets, each with different plausible imputations. You analyze each dataset and then combine the results in a way that reflects both within-imputation and between-imputation uncertainty.
That combination step matters. It prevents the analysis from acting as if the imputed values were known for sure.
Multiple imputation is not a universal solution, but it is often a much better default than deletion or single-value imputation when data are missing in a way that can be modeled from observed information.
The imputation model matters
Imputation is only as good as the model behind it.
If you impute a variable without using predictors that are related to it, the imputations can be poor. If you ignore important structure, such as groups, repeated measures, or bounds, the imputations can be less credible.
A good imputation model should usually include:
- variables related to the missing values,
- variables related to the chance of missingness,
- and any structure needed to preserve the analysis model’s relationships.
In other words, the imputation model should be at least as thoughtfully built as the analysis model.
Diagnostics are part of the job
Because missingness assumptions are not fully testable from the data you have, diagnostics are especially important.
Useful checks include:
- comparing observed and imputed distributions,
- checking whether imputed values land in plausible ranges,
- seeing whether conclusions change across imputation models,
- and exploring whether results are sensitive to different assumptions about missingness.
Sensitivity analysis is especially important when missing-not-at-random is plausible. If your conclusions depend heavily on an unverifiable assumption, you should say so.
A practical workflow
A sensible workflow looks like this:
- Quantify how much is missing and where.
- Look for patterns: is missingness concentrated in some groups, times, or variables?
- Think about why the data might be missing.
- Decide whether complete-case analysis, simple handling, or a more serious imputation approach is appropriate.
- If you impute, build an imputation model that reflects the analysis question.
- Check diagnostics and run sensitivity analyses.
- Report what was missing, how you handled it, and what you could and could not assume.
That last step is often skipped, but it is one of the most important.
What to avoid
Common missing-data mistakes include:
- deleting cases without checking whether the missingness is related to the outcome or predictors,
- using mean imputation as if it solved the problem,
- imputing from too few variables,
- ignoring the uncertainty that missingness creates,
- pretending the data are “missing at random” without thinking about the mechanism,
- and failing to report how much was missing or how it was handled.
A clean report does not hide these choices.
A worked sketch
Suppose a study has a baseline score, a follow-up score, and a few dropouts at follow-up.
If the dropout is unrelated to the outcome and the baseline data are rich, the missingness may be easier to handle. If participants with worse baseline scores are more likely to drop out, then complete-case analysis could bias the result upward.
In that situation, an appropriate imputation approach might use baseline scores and other observed variables to impute follow-up values, while reflecting uncertainty. A sensitivity analysis might ask what would happen if dropouts improved less, or more, than the observed pattern suggests.
The point is not to claim certainty. The point is to make the assumption explicit.
A second sketch
Suppose a survey has missing income data and people with lower education are less likely to answer.
If education predicts missingness, then the missingness is not completely random. Ignoring education in the imputation or analysis could leave a biased picture.
Here the emphasis should be on using the available information honestly, checking whether the imputation distribution makes sense, and reporting how much income data was missing and from whom.
Why this is not just a technical nuisance
Missing data is a scientific issue because it can change the story.
A result can look stronger, weaker, or different depending on how missingness was handled. That means missingness decisions should be visible, not buried in a footnote.
If a conclusion depends on assuming that the missing data behave themselves, that is exactly the kind of assumption a careful researcher should flag.
Key takeaways
- Missing data is a problem of assumptions, not just missing cells.
- The mechanism matters: completely at random, at random, or not at random.
- Complete-case analysis can bias results when missingness is related to the variables of interest.
- Mean imputation is usually too crude for analysis.
- Multiple imputation is often a better practical choice when used carefully.
- The imputation model should reflect the analysis structure and predictors.
- Sensitivity analysis matters when assumptions are uncertain.
- Always report how much was missing, from where, and how it was handled.
Challenge
Take a dataset or study you know and write a short missing-data note. State how much is missing, where it is missing, what mechanism seems most plausible, how you would handle it, and which sensitivity analysis you would run if the missingness mechanism were uncertain.
Handling missing data in a real analysis? Ampersand Academy offers one-to-one mentoring on statistics and data analysis — part of Mahadhi, a development and digital marketing company.
—
FAQ
Not always, but it is always worth examining. Small amounts of missingness may matter less, but the mechanism and pattern still determine whether it is safe to ignore.
- Is missing data always a big problem?
Missing at random means the chance of missingness depends on observed data and not on the missing value itself, after accounting for what you observed. Missing not at random means the missingness depends on the missing value or another unobserved factor.
- What is the difference between missing at random and missing not at random?
Sometimes, if the missingness is limited and unlikely to bias the result. But it should be a deliberate choice, not a default.
- Is complete-case analysis ever okay?
Because it treats the missing value as known, reduces variability, and can distort relationships and uncertainty.
- Why is mean imputation discouraged?
It is a method that fills in missing values several times with plausible values, analyzes each completed dataset, and combines the results to reflect imputation uncertainty.
- What is multiple imputation?
No. It creates plausible values based on the observed data and model. The goal is better inference, not perfect reconstruction.
- Does imputation recover the true missing values?
Report how much is missing, where it is missing, what you assume about the mechanism, how you handled it, and whether the conclusions are sensitive to those choices.
- What should I report about missing data?
Is missing data always a big problem?
Not always, but it is always worth examining. Small amounts of missingness may matter less, but the mechanism and pattern still determine whether it is safe to ignore.
What is the difference between missing at random and missing not at random?
Missing at random means the chance of missingness depends on observed data and not on the missing value itself, after accounting for what you observed. Missing not at random means the missingness depends on the missing value or another unobserved factor.
Is complete-case analysis ever okay?
Sometimes, if the missingness is limited and unlikely to bias the result. But it should be a deliberate choice, not a default.
Why is mean imputation discouraged?
Because it treats the missing value as known, reduces variability, and can distort relationships and uncertainty.
What is multiple imputation?
It is a method that fills in missing values several times with plausible values, analyzes each completed dataset, and combines the results to reflect imputation uncertainty.
Does imputation recover the true missing values?
No. It creates plausible values based on the observed data and model. The goal is better inference, not perfect reconstruction.
What should I report about missing data?
Report how much is missing, where it is missing, what you assume about the mechanism, how you handled it, and whether the conclusions are sensitive to those choices.