Deep learning for medical applications faces many unique challenges. A major challenge is the large amount of labelled data for training, while working in a relatively data scarce environment. Active learning can be used to overcome the vast data need challenge. A second challenged faced is poor performance outside of a experimental setting, contrary to the high requirement for safety and robustness. In this paper, we present a novel framework for estimating uncertainty metrics and incorporating a similarity measure to improve active learning strategies. To showcase effectiveness, a medical image segmentation task was used as an exemplar. In addition to faster learning, robustness was also addressed through adversarial perturbations. Using epistemic uncertainty and our framework, we can cut number of annotations needed by 39% and by 54% using epistemic uncertainty and a similarity metric.