Shortage of annotated data is one of the greatest bottlenecks related to deep learning in healthcare. Methods proposed to address this issue include transfer learning, crowdsourcing and self-supervised learning. More recently, first attempts to leverage the concept of meta learning have been made. Meta learning studies how learning systems can increase in efficiency through experience, where experience can be represented by solutions to tasks connected to previously acquired data, for example. A core capability of meta learning-based approaches is the identification of similar previous tasks given a new task. Quantifying the similarity between tasks, however, is an open research problem. We address this challenge by investigating two complementary approaches: (1) Leveraging images and labels to embed a complete data set in a vector of fixed length that serves as a task fingerprint (2) Directly comparing the distributions of the images with sample-based and optimal transport-based methods, thereby neglecting the labels.