Poster
in
Workshop: Statistical Frontiers in LLMs and Foundation Models
Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction
Drew Nguyen · Reese Pathak · Anastasios Angelopoulos · Stephen Bates · Michael Jordan
Keywords: [ confomal prediction ] [ decision-making ] [ tradeoffs ] [ empirical process theory ] [ risk control ]
Decision-making pipelines are generally characterized by tradeoffs among various risk functions. Itis often desirable to manage such tradeoffs in a data-adaptive manner. As we demonstrate, if this isdone naively, state-of-the art uncertainty quantification methods can lead to significant violations ofputative risk guarantees. To address this issue, we develop methods that permit valid control of riskwhen threshold and tradeoff parameters are chosen adaptively. Our methodology supports monotoneand nearly-monotone risks, but otherwise makes no distributional assumptions. To illustrate the benefitsof our approach, we carry out numerical experiments on synthetic data and the large-scale vision datasetMS-COCO.