Human visual classification is the traditional approach to identifying jellyfish galaxies. However, this approach is unsuitable for large-scale galaxy surveys. In this study, we employ self-supervised learning on a dataset of approximately 200 images to extract semantically meaningful representations of galaxies. Despite the small dataset size, a similarity search suggests that the self-supervised representation space contains meaningful morphological information. We propose a framework for assigning JClass, a categorical disturbance measure, based on nearest-neighbor search in the self-supervised representation space to assist visual classifiers. Our pipeline is highly adaptable, allowing for the seamless identification of any rare astronomical signatures within astronomical datasets.