Quick answer: The right regression model is mostly determined by the outcome you actually measured, not by a preference for the familiar linear model. Continuous outcomes on a sensible scale often suit linear regression. Binary outcomes call for a model like logistic regression. Counts call for something like Poisson or negative binomial regression. Rates, skewed continuous data, and bounded outcomes each have their own common options. Pick the model that matches the outcome and the question, then check whether its assumptions are reasonable.
Part 4 of a practical statistics series for researchers. If the basics are new, start with p-values, effect sizes, and confidence intervals.
Table of Contents
Start with the outcome, not the habit
A surprisingly common mistake is to run ordinary linear regression because it is familiar, even when the outcome is binary, a count, a rate, or a skewed score.
The outcome is not a decoration. It shapes:
- what scale the model works on,
- what the coefficients mean,
- what the errors should look like,
- and what predictions make sense.
If you use a model that does not fit the outcome, you can end up with predictions outside the possible range, coefficients that are hard to interpret, or standard errors that miss the structure of the data.
The first question is therefore: what kind of thing is the outcome?
Continuous outcomes
A continuous outcome is one that can take many values along a scale, such as height, time, cost, or a test score.
Linear regression is a natural starting point when:
- the outcome is roughly continuous,
- the relationship with predictors is roughly linear in the chosen scale,
- the residuals are not badly skewed or heteroscedastic,
- and predictions staying within the possible range is not a serious problem.
That last point matters. A linear model can predict negative values for a strictly positive outcome, or predict values beyond a known bound. If that is implausible, the linear model may still be useful as an approximation over a narrow range, but it is worth questioning.
For skewed continuous outcomes, a transformation can sometimes help. A log transformation is common when the outcome is positive and the effects look more like ratios than differences. But transformations also change interpretation, so choose deliberately rather than reflexively.
Binary outcomes
A binary outcome has two categories: yes/no, success/failure, remission/no remission, click/no click.
For these outcomes, linear regression is usually a poor default because it can predict values outside 0 and 1 and because the error structure is not appropriate for a two-category response.
A common alternative is logistic regression, which models the log odds of the outcome as a linear function of the predictors. That gives:
- predictions bounded between 0 and 1,
- odds ratios as a familiar way to describe associations,
- and a multiplicative structure on the odds scale.
Logistic regression is not the only option, but it is the workhorse for binary outcomes.
Do not confuse “logistic regression” with “linear regression with a binary outcome.” They answer the same kind of question in very different ways.
Count outcomes
A count outcome records how many times something happened: hospital visits, citations, defects, purchases, website visits.
A common first model is Poisson regression, which models the log of the expected count as a linear function of predictors. It assumes the mean and variance are related in a particular way.
That assumption is often too restrictive. Real count data are frequently more variable than Poisson expects, a problem called overdispersion.
When overdispersion is present, negative binomial regression is a common alternative because it allows more variability for a given mean. If the data include many zeros, other options such as zero-inflated or hurdle models may be worth considering, depending on why the zeros appear.
The main point is that count data should not usually be treated like continuous data with a linear model, especially when counts are small or the variance is large relative to the mean.
Rates and exposures
Sometimes the outcome is not just a count, but a count relative to exposure time, area, population size, or another denominator.
In those cases, the model should account for the exposure. A common approach is to include the exposure as a offset on the appropriate scale, so the model estimates a rate instead of a raw count.
That distinction matters because two groups with the same count can have very different rates if their exposure differs.
Which model for which question
A simple decision outline looks like this:
- If the outcome is continuous and the linear model is reasonable, start there.
- If the outcome is positive and skewed, consider a transformation or a model designed for positive skewed data.
- If the outcome is binary, use logistic regression or a related binary model.
- If the outcome is a count, start with Poisson or negative binomial regression.
- If the outcome is a rate, use a count model with an offset.
- If the outcome is ordered categories, consider ordinal models.
- If the outcome has multiple categories, consider multinomial models.
This is not a rigid menu. It is a starting checklist.
Interpretation changes with the model
One reason model choice matters is that it changes how you describe results.
- In linear regression, a coefficient is usually a difference in means or a slope on the original scale.
- In logistic regression, the coefficient is on the log odds scale, and the exponentiated coefficient is an odds ratio.
- In Poisson regression, the exponentiated coefficient is a rate ratio.
- In negative binomial regression, you also work on a ratio scale, but the variance structure is different.
Readers should not have to guess which scale you used. State it plainly.
Why the linear model feels tempting
The linear model is tempting because it is simple and widely understood. That simplicity is a real advantage when it fits.
But simplicity is only an advantage when it does not mislead. A linear model can be deceptive when the outcome is bounded, discrete, skewed, or strictly non-negative. In those cases, the model can give predictions and inference that look clean while being inappropriate.
A good model is not the most complex one or the simplest one. It is the one that matches the data and the question closely enough to be trustworthy.
Model choice is not the same as model checking
Choosing a model is only the first step. After that, you still need to check whether the model is doing what you expect.
For linear regression, this often means looking at residuals, checking for non-linearity, heteroscedasticity, and influential points.
For GLM-type models, this often means checking whether the distribution fits, whether the link function is reasonable, and whether there is extra variability or unexpected structure.
A model can be “the right type” in principle and still be a poor fit in practice.
A practical example
Suppose you are studying whether an intervention changes the number of support tickets per customer.
If you model tickets as continuous with linear regression, you may get predictions below zero or an awkward fit for customers with zero or one ticket.
If you model tickets as counts, you may get a better match. If the variance is larger than the mean, a negative binomial model may fit better than Poisson. If many customers have zero tickets, you may need to think about why and whether a zero-inflated model is appropriate.
The choice depends on the outcome and on what you want to estimate.
A second example
Suppose you are studying whether a treatment affects whether patients achieve remission.
If you model remission as a continuous score when it is actually a yes/no outcome, the model is mismatched. If remission is binary, logistic regression is a more natural choice, and the result is usually reported as an odds ratio with a confidence interval.
If you want a risk difference instead of an odds ratio, that is possible too, but you should choose the measure that matches your question and say so clearly.
Common mistakes
- Using linear regression by default for binary or count outcomes.
- Treating odds ratios as if they were risk ratios without checking the context.
- Ignoring overdispersion in count data.
- Ignoring that a skewed outcome may need a different scale or model.
- Forgetting that transformations change interpretation.
- Choosing the model based on p-values alone rather than on fit and question.
- Not checking residuals or predictions after fitting.
What to report
When you report a regression result, say:
- the outcome type,
- the model family,
- the main effect measure,
- the coefficients or exponentiated coefficients,
- the uncertainty intervals,
- and any important limitations in fit.
If the model is not the usual default for the outcome, explain why you chose it.
Key takeaways
- Choose the model based on the outcome type and the question.
- Linear regression suits many continuous outcomes, but not all.
- Binary outcomes usually need logistic regression or a related model.
- Count outcomes often need Poisson or negative binomial regression.
- Rates need models that account for exposure.
- Model choice changes interpretation.
- A reasonable model still needs to be checked.
Challenge
Take a dataset or study idea you have and write a one-paragraph model choice justification. State the outcome type, the model you would start with, the effect measure you would report, and at least one assumption you would check before trusting the result.
Choosing models for real research data? Ampersand Academy offers one-to-one mentoring on statistics and data analysis — part of Mahadhi, a development and digital marketing company.
—
FAQ
Because the outcome type matters. Linear regression can be a poor fit for binary, count, or highly skewed outcomes, and it can produce predictions or inference that do not match the data.
- Why not just use linear regression for everything?
When the outcome is binary and you want to model the probability of one category. It gives odds ratios and keeps predictions between 0 and 1.
- When is logistic regression appropriate?
It is when the observed variability is larger than a model like Poisson expects. If ignored, standard errors and inference can be off.
- What is overdispersion?
Not always. Transformations can help, but they also change interpretation. Sometimes a model designed for the outcome type is better than transforming a continuous approximation.
- Should I always transform a skewed outcome?
Start from the outcome type and the question, then check fit and assumptions. “Best” usually means “most appropriate and checkable,” not “the one with the prettiest p-value.”
- How do I know which model is best?
No. Model choice should consider fit, interpretability, and assumptions, not only whether one term is significant.
- Can I compare models with p-values alone?
Many outcomes need more specialized models, such as ordinal, multinomial, survival, or mixed models. The same principle applies: match the model to the outcome and question.
- What if my outcome is neither continuous, binary, nor a simple count?
Why not just use linear regression for everything?
Because the outcome type matters. Linear regression can be a poor fit for binary, count, or highly skewed outcomes, and it can produce predictions or inference that do not match the data.
When is logistic regression appropriate?
When the outcome is binary and you want to model the probability of one category. It gives odds ratios and keeps predictions between 0 and 1.
What is overdispersion?
It is when the observed variability is larger than a model like Poisson expects. If ignored, standard errors and inference can be off.
Should I always transform a skewed outcome?
Not always. Transformations can help, but they also change interpretation. Sometimes a model designed for the outcome type is better than transforming a continuous approximation.
How do I know which model is best?
Start from the outcome type and the question, then check fit and assumptions. Best usually means most appropriate and checkable, not the one with the prettiest p-value.
Can I compare models with p-values alone?
No. Model choice should consider fit, interpretability, and assumptions, not only whether one term is significant.
What if my outcome is neither continuous, binary, nor a simple count?
Many outcomes need more specialized models, such as ordinal, multinomial, survival, or mixed models. The same principle applies: match the model to the outcome and question.