Logistic Regression Explained: Odds Ratios, Probabilities, and What Coefficients Mean

You have an outcome with two values: responded or did not, diseased or healthy, converted or churned. You also have predictors. The obvious move is linear regression, and it fails immediately — a straight line predicts values below zero and above one, which are not probabilities.

Logistic regression fixes this by modelling the log-odds of the outcome instead of the outcome itself. That single change is what makes the coefficients interpretable as odds ratios, and it is also the source of most of the confusion around them.

Why a straight line fails for a yes/no outcome

Fit a linear model to binary data and three things go wrong at once. Predictions fall outside [0, 1] for extreme predictor values. The residuals are not normally distributed — they can only take two values for each fitted value. And the variance is not constant: it is largest near a probability of 0.5 and vanishes at the extremes, which violates homoscedasticity.

None of that is fatal for a rough description. It is fatal when you want a probability, a confidence interval, or a likelihood you can compare between models.

What the model actually estimates

Logistic regression models the log-odds, also called the logit, as a linear function of the predictors:

logit(p) = ln( p / (1 - p) ) = b0 + b1*x1 + b2*x2 + ...

The inverse of that mapping is the logistic function, which squeezes any real number into (0, 1): p = 1 / (1 + exp(-(b0 + b1*x1 + ...))). So the model is linear on the log-odds scale and curved on the probability scale — the S-shape you have seen a thousand times.

Scale	Range	Why it matters
Probability	0 to 1	What a reader wants; bounded, so unsuitable for a linear model
Odds	0 to infinity	Ratio of success to failure; still bounded at zero
Log-odds (logit)	minus to plus infinity	Where the linear model lives and coefficients are additive

That table is the whole trick. On the log-odds scale, adding one unit of a predictor adds a constant amount to the log-odds — regardless of where you start. On the probability scale the same change moves p a lot near 0.5 and very little near 0 or 1.

Reading coefficients as odds ratios

Coefficients from a logistic model are on the log-odds scale, which nobody finds intuitive. Exponentiating them gives odds ratios, which are intuitive but easy to misstate.

# R
m <- glm(responded ~ dose + age, data = trial, family = binomial)
summary(m)
exp(coef(m))                 # odds ratios
exp(confint(m))              # 95% CI on the odds-ratio scale

An odds ratio of 1.8 for dose means: one unit higher dose multiplies the odds of responding by 1.8. Two statements are wrong and both are common.

1. It is not “an 80% higher probability”. Odds and probability diverge sharply, especially when the outcome is common.
2. It is not a constant probability difference. The odds ratio is constant by construction; the probability difference depends on the starting probability.

A worked contrast makes the gap concrete. Suppose the baseline probability is 10%, so the odds are 0.111. Multiply by 1.8 to get 0.2, then convert back: p = 0.2 / 1.2 = 16.7%. A 6.7 percentage-point gain. Now start at 50%: odds 1.0 becomes 1.8, so p = 64.3% — a 14.3 point gain. Same odds ratio, more than double the probability effect.

Baseline p	Baseline odds	Odds x1.8	New p	Change
10%	0.11	0.20	16.7%	+6.7 pts
30%	0.43	0.77	43.5%	+13.5 pts
50%	1.00	1.80	64.3%	+14.3 pts
80%	4.00	7.20	87.8%	+7.8 pts

This is why a good paper reports both the odds ratio and predicted probabilities at meaningful values of the predictors. The odds ratio is stable and comparable; the probabilities are what a decision-maker actually needs.

Fitting it in R and Python

In R the base glm with family = binomial is the reference implementation. In Python, statsmodels gives you the inferential output you want, while scikit-learn gives you prediction and regularisation.

# Python — inference
import statsmodels.api as sm

X = sm.add_constant(df[["dose", "age"]])
m = sm.Logit(df["responded"], X).fit()
print(m.summary())
print(np.exp(m.params))                      # odds ratios
print(np.exp(m.conf_int()))                  # CI on OR scale

# Python — prediction with regularisation
from sklearn.linear_model import LogisticRegression

clf = LogisticRegression(penalty="l2", C=1.0, max_iter=1000)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)[:, 1]

A warning about scikit-learn for inference: by default it applies L2 regularisation, which shrinks coefficients and therefore biases odds ratios toward 1. If you want interpretable coefficients, either set penalty=None or use statsmodels.

Predicted probabilities

newdata <- data.frame(dose = c(10, 20, 30), age = 45)
predict(m, newdata, type = "response")   # probabilities, not log-odds

Forgetting type = "response" is the single most common R mistake here: you get log-odds and read them as probabilities, which reports absurd numbers like 2.4 or -1.1 and is occasionally published anyway.

Checking the model is not lying

Logistic regression makes fewer assumptions than linear regression — no normality, no homoscedasticity — but it still assumes independent observations, a linear relationship on the log-odds scale, and no severe separation or collinearity.

Check	What it catches	How
Complete separation	A predictor that perfectly splits the outcome; coefficients explode	Inspect for huge SEs; use Firth or penalised likelihood
Linearity on log-odds	A continuous predictor whose effect is U-shaped	Spline or binned residuals plot
Influential points	One row driving the whole result	Cook distance, dfbeta
Calibration	Scores rank well but are systematically too high or low	Calibration plot, Hosmer-Lemeshow
Discrimination	Whether ranking is any good at all	ROC / AUC

# Calibration and discrimination, quickly
p <- predict(m, type = "response")
mean((p - trial$responded)^2)                     # Brier score, lower is better

library(pROC)
auc(trial$responded, p)                          # discrimination

Discrimination and calibration are different questions, and a model can pass one while failing the other. AUC asks whether cases rank above non-cases; calibration asks whether a predicted 30% really means 30 in 100. For clinical or business use, calibration is usually the more consequential of the two.

Multicollinearity and sample size

Two highly correlated predictors split their effect between them, producing large standard errors and unstable odds ratios. Variance inflation factors above about 5 are a warning. Sample size is the other quiet constraint: a rough rule of thumb is at least ten events per predictor, where events means the smaller of the two outcome counts — not the total sample size.

# 200 patients, 30 events, 4 predictors -> underpowered and unstable
table(trial$responded)

What to report

A defensible logistic regression report contains: the model formula with the link stated, odds ratios with 95% confidence intervals, predicted probabilities at meaningful predictor values, a calibration statement, and the sample size with event counts. Add the AUC only if discrimination is genuinely part of the question.

For a reader who wants the decision-theoretic view — how a probability turns into a threshold and a decision — the natural next step is Ampersand Academy, which covers model evaluation end to end.

Practice

1. Fit a logistic model with one continuous predictor and state the odds ratio in one sentence without using the word “probability”.
2. Convert that odds ratio into predicted probabilities at three baseline risks and describe how the effect size changes.
3. Re-fit with scikit-learn using penalty=None and confirm the coefficients match statsmodels.
4. Produce a calibration check and decide whether the model is usable, separately from its AUC.

Related tutorials

• Choosing the Right Regression Model: Linear, GLM, and When Each One Fits — why a binary outcome needs a link function instead of a straight line
• Multiple Comparisons and False Discovery Rate: What Changes When You Test a Lot — adjusting when you test many predictors at once
• ROC Curves and AUC: Evaluating Diagnostic Tests and Classifiers — measuring how well the fitted model separates the two outcomes