Causal inference for clinical and observational data
Potential-outcomes and graphical frameworks, propensity and doubly robust estimators, target trial emulation and sensitivity analysis for estimating treatment effects from EHR and registry data.
Real-world evidence, regulatory use of observational data, treatment-effect heterogeneity and learning clinical policies all require identification assumptions and estimators from this theory.
Statistical methods for working out what a treatment really does from messy real-world patient data.
1Observational patient records
2Sicker patients differ in treatment
3Adjust for recorded confounders
4Emulate the ideal trial
5Bound the effect of hidden ones
Setup. Each patient has potential outcomes Y(1) and Y(0) under treatment and no treatment. We never see both. The average effect is τ=E[Y(1); the conditional effect depends on patient features .
Go deeper
The full technical text.
What it is
Estimating what would happen under a treatment from data in which treatment was not randomly assigned. Each patient has potential outcomes Y(1),Y(0); the individual effect Y(1)−Y(0) is never observed, so we target the average effect or the conditional effect (CATE).
32 trials emulated in claims data; agreement depended strongly on how closely the trial design could be emulated (later corrected to 72 percent significance agreement).
by · AI-reviewed
Discussion
−
Y(0)]
τ(x)
x
Identification. If treatment is as good as random given recorded features (no hidden confounding), if every kind of patient could have received either treatment (positivity), and outcomes are consistent, then τ equals the difference between the outcome averages when we standardise to the same patient mix. Each condition can fail in hospital data.
Estimators. Regression of outcome on treatment and features; inverse-probability weighting by the propensity score e(x)=P(A=1∣X=x); doubly robust estimators that are right if either of the two models is right; double machine learning, which uses sample splitting so flexible algorithms can model the nuisance parts without spoiling inference; causal forests and meta-learners for effects that vary between patients. For treatments that change over time, Robins' g-methods handle feedback between treatment and health.
Design. Target trial emulation fixes eligibility, the start of follow-up, treatment strategies and outcomes, which removes common biases such as immortal time. In the RCT-DUPLICATE project, emulations of 32 trials agreed in statistical significance in about 72 percent of cases and agreed closely when the trial design was copied well.
Sensitivity analysis. Because no data can prove there is no hidden confounder, we bound the effect assuming confounding up to a stated strength. Recent work gives sharp bounds and conformal prediction ranges, but calibrating the strength remains open.
Skills. Graphs and the backdoor rule, semiparametric efficiency, cross-fitting, overlap diagnostics.
τ=E[Y(1)−Y(0)]
τ(x)=E[Y(1)−Y(0)∣X=x]
Identification
Under unconfoundedness (Y(a)⊥A∣X), positivity (0<P(A=1∣X)<1) and consistency, τ is identified by the g-formula E[E[Y∣A=1,X]−E[Y∣A=0,. Each assumption can fail in electronic health records: hidden confounders (frailty, indication, unrecorded behaviour), near-deterministic treatment rules, and time-varying confounders affected by past treatment.
Estimators
Outcome regression; inverse-probability weighting with the propensity score e(x)=P(A=1∣X=x) (Rosenbaum-Rubin 1983); doubly robust augmented IPW and targeted maximum likelihood; double/debiased machine learning (Chernozhukov et al. 2018), which uses cross-fitting so flexible ML can estimate nuisance functions while τ^ stays root-n consistent; meta-learners (S, T, X, R) and causal forests (Wager-Athey 2018) for CATE. For longitudinal treatment, Robins' g-methods (marginal structural models, g-computation, structural nested models).
Design layer
Target trial emulation (Hernán and Robins 2016) writes the hypothetical randomised trial (eligibility, time zero, strategies, outcome, follow-up) and emulates it in the data, avoiding immortal-time and selection biases. The RCT-DUPLICATE initiative emulated 32 trials in claims databases: significance agreement was 72 percent after a 2023 correction, and correlation was high (r about 0.94) when emulation was close.
Sensitivity analysis
Because unconfoundedness is untestable, one bounds the effect under a stated violation: Rosenbaum bounds, the marginal sensitivity model (Tan 2006), E-values, and recent sharp bounds with statistical guarantees (Dorn-Guo-Kallus; Frauen et al.; Jin-Ren-Candès conformal individual-effect intervals).
Techniques to master
Directed acyclic graphs and the backdoor criterion, semiparametric efficiency theory, cross-fitting, overlap diagnostics, conformal prediction, negative controls and benchmarking against trials.
Where it bites
The open problem here is honest, tight, valid uncertainty for individual or subgroup treatment effects from EHR when unmeasured confounding cannot be ruled out.
What to read before Causal inference for clinical and observational data (left), and what builds on it: entries that depend on it and the open problems it hosts (right). Every box links to its page.