Absorption near edge structures in the core electron excitation spectra reflect the anisotropy of orbitals in the transition final state and can be used for analyzing local atomic environment including its orientation. So far, the analysis of fine structures is mainly based on a fingerprint-matching with high-cost experimental or simulated spectra. If core electron excitation spectra, including its anisotropy, can be predicted at low cost using machine learning, the application range of the core electron excitation spectra will be accelerated and extended for such as orientation and electronic structure analysis of liquid crystals and organic solar cells at high spatial resolution. In this study, we introduce a message-passing neural network for predicting core electron excitation spectra using a unit direction vector in addition to molecular graphs as input. Utilizing a database of calculated C K-edge spectra, we have confirmed that the network can predict core electron excitation spectra reflecting the anisotropy of molecules. Our model is expected to be expanded to other physical quantities in general that depend not only on molecular graphs but also on anisotropic vectors.