Uncertainty quantification for imaging-related inverse problems is drawing much attention lately. Existing approaches towards this task define uncertainty regions per pixel while ignoring spatial correlations. In this paper we propose PUQ (Principal Uncertainty Quantification) -- a novel definition of uncertainty that takes into account spatial relationships within the image, thus providing reduced uncertainty volume. Leveraging diffusion models, we derive uncertainty intervals around principal components of the empirical posterior distribution, accompanied by probabilistic guarantees. The proposed approach can operate globally on the entire image, or locally on patches, resulting in informative and interpretable uncertainty regions. We verify our approach on several inverse problems, showing a significantly tighter uncertainty regions compared to baseline methods.