Valid and non-conservative treatment-effect intervals from EHR under unmeasured confounding
Strong partial results exist for averages (doubly valid, doubly sharp bounds) and marginal coverage, but conditional validity, near-oracle sharpness and a principled choice of Gamma together are unsolved, and Gamma is fundamentally non-identifiable: research-grade at 55, below the virtual-cell problem (62). Application value 57 for real-world evidence and personalised decisions, below affinity (76); leverage 49 as it shapes causal ML for health data.
The problem
Giving honest, usable error bars on a treatment's effect for a patient when some important factors were never recorded.
- 1Treatment given for hidden reasons
- 2Assume bounded hidden influence
- 3Compute range of possible effects
- 4Make it hold for each patient type
- 5Choose the bound strength honestly
Setup. Observe with covariates, treatment and outcome. Potential outcomes ; target or the individual effect .
Start with Causal inference for clinical and observational data. No foundations are linked to it yet; if you know what a reader needs first, say so in the discussion below so a maintainer can link it.
State of the art
- Sharp bounds for averages. Dorn, Guo and Kallus (arXiv:2112.11449; JASA 2025) derive sharp bounds on the average treatment effect under the marginal sensitivity model and estimators that are "doubly valid" (bounds valid even if most nuisance functions are wrong) and "doubly sharp" (efficient when they are right).
- Sharp bounds for conditional effects. Frauen, Melnychuk and Feuerriegel (NeurIPS 2023, arXiv:2305.16988) generalise the marginal sensitivity model and derive sharp bounds for CATEs, mediation and distributional effects with a scalable estimator; guarantees are asymptotic and rely on correctly specified or sufficiently accurate nuisance estimators.
- Individual effects. Jin, Ren and Candes (PNAS 2023; arXiv:2111.12161) give a conformal prediction method with marginal coverage under the marginal sensitivity model, and a Gamma-value per unit (the minimum confounding that explains the effect away). Their guarantees are marginal over covariates, not conditional on , and a concurrent method by Yin, Shi, Wang and Blei (arXiv:2112.03493) is reported as equivalent.
- Anchoring to trials. RCT-DUPLICATE (JAMA 2023) emulated 32 trials in claims data: significance agreement was 75% originally and 72% after a published correction, with correlation about 0.94 for closely emulated trials; this supplies empirical but dataset-specific information about how large real-world confounding is.
- Not shown. No procedure combines conditional validity, near-oracle sharpness and a principled choice of in EHR-scale data with weak overlap in sources reviewed (October 2026).
Realistic applications
Clinicians and regulators need to know how far a data-driven recommendation for a specific patient group can be trusted. Valid, informative intervals would support real-world evidence submissions, guide which subgroups merit a trial, and prevent harmful policy learning from confounded records.
What counts as solved
A refereed method with theorems for items 1-4 and an empirical validation on at least five trial-anchored EHR emulations (where the RCT effect is the hidden truth) showing nominal coverage for the individual-effect and CATE intervals and widths within 1.2 times the oracle bound where computable, together with open software.
How it gets verified
ReproductionA training run, experiment or computation rerun by someone else.
A refereed method with theorems for items 1 to 4 must show nominal coverage for individual-effect and CATE intervals on at least five trial-anchored EHR emulations (the RCT effect is the hidden truth), with widths within 1.2 times the oracle bound where computable, and release open software.
Claims under verification
+ add a claimNo claimed results on the Verification board for this problem yet.
Where the work is
A statistics-theory problem, validated with trial-anchored EHR emulations.
Bottlenecks
ranked by votes- #1TheoryThe confounding strength Gamma is not identifiable
The confounding strength is an assumption; choosing it from data requires external information (negative controls, trials).
Community grades
Sign in to add your own grade.
Connected problems
No connections yet.
Progress
- Jan 1, 2025Doubly valid and sharp bounds in JASAsource
Dorn, Guo and Kallus' estimators published in the Journal of the American Statistical Association.
- May 26, 2023Sharp CATE bounds (NeurIPS)source
Frauen et al. derive sharp bounds for CATEs and other quantities under a generalised sensitivity model.
- Nov 23, 2021Conformal sensitivity analysis for ITEs (preprint)source
Jin, Ren and Candes give marginally valid intervals for individual treatment effects under the marginal sensitivity model.
- Dec 1, 2006Marginal sensitivity modelsource
Tan introduces a bound on the odds-ratio between propensity scores with and without the unmeasured confounder.