Poster
in
Workshop: Statistical Frontiers in LLMs and Foundation Models
Conformal Prediction Adaptive to Unknown Subpopulation Shifts
Nien-Shao Wang · Sai Praneeth Karimireddy
Keywords: [ provable uncertainty quantification ] [ conformal prediction ] [ distribution shift ] [ subpopulation shift ]
Conformal prediction is widely used to provide uncertainty quantification for black-box machine learning models with formal coverage guarantees. However, these guarantees typically break down in the presence of distribution shifts, where the data distribution at test time differs from the training distribution. In this work, we address subpopulation shifts, where the test environment exhibits an unknown and differing mixture of subpopulations compared to the training data. We propose a method to adapt conformal prediction to these unknown subpopulation shifts. Our results demonstrate that the proposed algorithms maintain coverage in test domains where standard conformal prediction fails.