Abstract: We report the discovery of four latent variables that effectively capture the complexity of high-dimensional galaxy spectra from the Sloan Digital Sky Survey. We investigate the spectral ranges and the physical properties of galaxies that are most informative for explaining the observed spectra. Employing Variational Autoencoders (VAEs) and conditional VAEs, both being generative models proficient at capturing intricate details in high-dimensional data, we show that these four latent parameters provide more information than traditionally utilized physical properties such as stellar mass, Star Formation Rate, specific Star Formation Rate, and metallicity. We highlight that the spectral features providing the most insight include the range below 5000\AA and the wavelengths corresponding to the emission lines ([O II], [O III], and H$\alpha$). Our results indicate that we can construct a more efficient representation of galaxy spectra based on these latent parameters, which are more fundamental than currently acknowledged physical properties.