Causal Inference from Observational Data: Confounding, Propensity Scores and Sensitivity

Everyone who has taken a statistics course can recite that correlation is not causation. Far fewer can say what would be required to get from one to the other using observational data — which is the situation for most of the data we actually have. Randomised trials are the exception, and most questions do not get one.

Causal inference is the discipline of making that jump explicitly rather than accidentally. It does not remove the need for assumptions; it names them, so a reader can attack them.

What the dictum actually means

Two variables are associated when they move together. One causes the other when intervening on the first would change the second. Those are different claims, and an association can arise from three distinct structures: X causes Y, Y causes X, or a third variable Z causes both.

The third case is confounding, and it is the reason observational estimates can be biased in either direction. A regression coefficient describes an association conditional on the variables you included. Whether it approximates a causal effect depends entirely on whether the right variables were among them.

Draw the DAG before you model anything

A directed acyclic graph makes your assumptions visible: nodes are variables, arrows are direct effects. You do not need software to draw one; you need to be willing to commit to a structure and let someone disagree with it.

# A minimal example, expressed with dagitty
library(dagitty)
g <- dagitty("dag {
  age -> treatment
  age -> outcome
  severity -> treatment
  severity -> outcome
  treatment -> outcome
}")
adjustmentSets(g, exposure = "treatment", outcome = "outcome")

The output tells you which variables must be adjusted for to block every backdoor path. That is the entire purpose of the exercise: it converts a vague fear about confounders into a finite, testable list.

The collider rule catches people out. Adjusting for a variable that is caused by both treatment and outcome induces an association where none exists — conditioning on hospital admission, for instance, when treatment affects admission and admission relates to outcome. More adjustment is not safer adjustment.

Randomisation as the reference standard

Randomisation makes treatment independent of every confounder, measured or not, which is why it needs no DAG to be credible. Observational methods are attempts to approximate that independence using measured variables, and they inherit a fatal weakness: they cannot adjust for what nobody recorded.

That is not a reason to give up. It is a reason to report a sensitivity analysis, which we will come back to.

Propensity scores

The propensity score is the probability of receiving treatment given covariates. Its useful property is that, among subjects with the same propensity score, treatment assignment is independent of those covariates — so comparing treated and untreated subjects with similar scores approximates a randomised comparison on the measured variables.

# R
library(MatchIt)
ps <- matchit(treatment ~ age + severity + comorbidity,
              data = obs, method = "nearest", ratio = 1,
              caliper = 0.2, std.caliper = TRUE)

summary(ps)              # check balance, not model fit
matched <- match.data(ps)

Balance is what matters, not the AUC of the propensity model. A model that predicts treatment perfectly is useless, because treated and untreated subjects then have no overlap. Check standardised mean differences before and after matching — a common target is below 0.1 for every covariate.

library(cobalt)
love.plot(ps, thresholds = c(m = 0.1))

The overlap assumption

If some treated subjects have a propensity score of 0.99 and no comparable untreated subject exists, no matching method can invent one. Trimming to the region of common support is not data dredging — it is acknowledging where a comparison is possible at all.

Weighting, and the doubly robust option

Inverse probability weighting reweights the sample so that the treated and untreated groups resemble each other. It uses all the data rather than discarding unmatched subjects, at the cost of sensitivity to extreme weights.

# IPW with a marginal structural-style estimate
obs$w <- ifelse(obs$treatment == 1, 1 / obs$ps, 1 / (1 - obs$ps))
obs$w <- pmin(obs$w, quantile(obs$w, 0.99))   # trim extreme weights

library(survey)
design <- svydesign(ids = ~1, weights = ~w, data = obs)
svyglm(outcome ~ treatment, design = design)

Trimming the top 1% of weights is a judgement call that should be reported. Untrimmed weights can let a handful of subjects dominate the estimate, producing a precise-looking result driven by three people.

Doubly robust estimators combine an outcome model with a treatment model, and remain consistent if either one is correctly specified. In R, MatchIt plus WeightIt cover the estimation, and targeted learning packages such as tmle and SuperLearner implement the modern versions.

The problem you cannot regress away

Unmeasured confounding is the central threat. No estimator detects it, and no amount of covariate adjustment addresses it. What you can do is quantify how strong it would have to be to overturn the result.

library(EValue)
# How strong would an unmeasured confounder need to be?
evalues.RR(est = 1.8, lo = 1.3, hi = 2.5)

An E-value of 2.2 means an unmeasured confounder would need to be associated with both treatment and outcome by a risk ratio of at least 2.2 to explain away the observed effect. That is a claim a reader can evaluate against domain knowledge — far more informative than a footnote saying residual confounding cannot be excluded.

Design choices that beat statistical fixes

Several study designs reduce confounding structurally rather than analytically. A new-user active-comparator design compares people starting two different treatments, avoiding the healthy-user bias of prevalent-user cohorts. Difference-in-differences uses a pre-period to absorb time-invariant confounding. Instrumental variables exploit a variable that affects treatment but not outcome except through it, at the cost of strong assumptions.

These are not exotic. Choosing the right design before collecting data is worth more than any estimator applied afterwards.

What to report

State the causal question in terms of an intervention, present the DAG and the adjustment set it implies, describe the estimator and why, show covariate balance (not model fit), report the effect with a confidence interval, and include a sensitivity analysis for unmeasured confounding. Say explicitly what would falsify your interpretation.

Causal claims are arguments, and the statistics are one component of them. If you want to build the design side of that argument — randomisation, blinding, comparator choice — Ampersand Academy covers it.

Practice

1. Draw a DAG for a question you care about and list the adjustment set it implies.
2. Identify a variable in your data that is a collider, and explain what adjusting for it would do.
3. Match on a propensity score and report standardised mean differences before and after.
4. Compute an E-value for a published observational effect and judge whether the required confounder is plausible.

Related tutorials

• Missing Data: Stop Treating Missing Like a Minor Detail — missingness as another assumption that shapes the estimate
• Checking Statistical Assumptions in Practice: What to Plot, What to Trust, What to Do Next — which assumptions are testable and which must be argued
• Logistic Regression Explained: Odds Ratios, Probabilities, and What Coefficients Mean — the outcome and treatment models behind propensity scores