The IceCube Neutrino Observatory at the South Pole is a cubic kilometer of Antarctic ice, instrumented with 5,160 digital optical modules. These modules collect light induced by neutrino interactions in the ice. This data is then used to identify the neutrino directions, their energies, and types, which are essential inputs for both particle physics and astrophysics. Deep learning methods, such as graph neural networks, have been successfully applied to the steady stream of incoming data IceCube is receiving. In this work, we build a foundation model on the IceCube data in a self-supervised way without any data labeling. This pre-trained model can be fine-tuned for the downstream task of directional reconstruction of neutrino events in a sample-efficient way.