Supervised dimensionality reduction seeks to map class-conditional data to a low-dimensional feature space while maximizing class discriminability. Although differences in class-conditional second-order statistics can often aid discriminability, most supervised dimensionality reduction methods focus on first-order statistics. Here, we present Supervised Quadratic Feature Analysis (SQFA), a dimensionality reduction technique that finds a set of features that preserves second-order differences between classes. For this, we exploit a relation between class discriminability and the Information geometry of second-moment (or covariance) matrices as points on the symmetric positive definite (SPD) manifold. We discuss the reasoning behind the approach, and demonstrate its utility in a simple vision task.