Reliability in medical image diagnostics is a required trait for any artificial system. Currently, most approaches rely on highly trained and specific models to leverage the feature quality learned from a particular type of medium, such as X-rays, NMR, PET scans and others. While this approach aligns with the standard human expert perspective, it also limits artificial systems to the representations learned from the dataset distribution. To gain a better understanding of how different media affect specific tasks, we explore task-specific feature transfer between domains. In this work, we propose the possibility of merging features from various areas to harness feature transfer in outlier cases. For this purpose, we develop an Algorithm Selection (AS) method that chooses algorithms trained on different sets of medical images and for different classification tasks. The AS system is then applied to a different classification task. The AS represents a set of methods that, given a problem and a range of existing algorithms, selects the best algorithm on a case- by-case basis. The results demonstrate the advantages of incorporating algorithms from different tasks and datasets in a supervised manner. By considering algorithms trained on diverse datasets, we can effectively capture outliers that might otherwise be neglected by more specific algorithms.