Abstract: We consider statistical inference under privacy constraints. In particular, we give differentially private algorithms for estimating coverage probabilities and computing valid confidence sets, and prove upper bounds on the error of our estimates and the length of our confidence sets. Our bounds apply to broad classes of data distributions and statistics of interest, and for fixed $\varepsilon$ we match the higher-order asymptotic accuracy of the standard (non-private) non-parametric bootstrap.