When considering off-policy reinforcement learning methods for treatment policies in healthcare data, it is generally the case that the patient population is diverse and has different chronic conditions that we would like to take into account when identifying optimal treatment policies. In this work, we use multi-group Gaussian process regression models in a fitted Q-iteration framework to allow us to model these different patient subgroups and adapt the optimal policies to each subgroup while estimating these function across the entire patient population. We apply our multi-group reinforcement learning (MGRL) framework to the problem of optimal treatment policies for women in childbirth with pre-existing conditions and across ethnicities to show the performance against other state-of-the-art methods