Poster
in
Workshop: Symmetry and Geometry in Neural Representations (NeurReps)
Training shapes the curvature of shallow neural network representations
Jacob Zavatone-Veth · Julian Rubinfien · Cengiz Pehlevan
Keywords: [ Neural Networks ] [ Riemannian geometry ] [ Curvature ]
We study how training shapes the Riemannian geometry induced by neural network feature maps. At infinite width, shallow neural networks induce highly symmetric metrics on input space. Feature learning in networks trained to perform simple classification tasks magnifies local areas and reduces curvature along decision boundaries. These changes are consistent with previously proposed geometric approaches for hand-tuning of kernel methods to improve generalization.