Neural networks; both biological and artificial, are commonly represented as graphs with connections between neurons, yet there is little understanding of the relationship between their graph structure and computational properties. Neuroscientists are trying to answer this question in biological neural networks or connectomes; however, there is a big opportunity to explore this in the vast domain of artificial neural networks. We present StructureReps, an architecture-agnostic framework for encoding neural networks as graphs using graph representation learning. By capturing key structural properties, StructureReps reveals strong correlations between network structure and task performance across various architectures. Additionally, this framework has potential applications beyond the decoding of neural network properties.