Abstract: High-Purity Germanium (HPGe) detectors represent a leading technology in the search for neutrinoless double beta ($0 \nu \beta \beta$) decay, a Beyond the Standard Model (BSM) process that, if discovered, would fundamentally revise our understanding of the universe. A key analysis task in $0 \nu \beta \beta$ decay experiments is the classification task of separating signals from background noise, which is traditionally approached as a supervised learning problem. However, HPGe detector data is often unlabeled, and producing ground-truth labels for every data point is an involved process. In this work, we reformulate the classification task of HPGe detector data as a weakly-supervised learning task and apply an episodic reinforcement learning (RL) algorithm with Randomized Return Decomposition (RRD) to address it. We evaluate our algorithm on real data produced by the MAJORANA DEMONSTRATOR HPGe detector experiment. With significantly fewer labels, the RL-trained weakly-supervised classifier slightly outperforms a fully-supervised classifier under the same energy cut. This work shows potential for training classifiers to reject background in future HPGe experiments like LEGEND.