Self-supervised learning (SSL) is a rich framework for obtaining meaningful data representations across large datasets. While SSL shows impressive results in computer vision and natural language processing, the single-cell field's diverse applications still need to be explored. We study SSL for the application of cell classification in cellular neighborhoods of spatially-resolved single-cell RNA-sequencing data. To address this, we developed an SSL framework on spatial molecular profiling data, integrating a cell's molecular expression and spatial location within a tissue slice. We demonstrate our methods on a large-scale whole mouse brain atlas, recording the gene expression measurements of 550 genes in 4,334,174 individual cells across 59 discrete tissue slices from the entire mouse brain. Our empirical study suggests that SSL improves downstream performance, especially in the presence of class imbalances. Notably, we observe a more substantial performance improvement on the sub-graph level than the full-graph level.