Current generative models for drug discovery primarily use molecular docking to guide the generation of active compounds. However, such models are often not useful in practice because even compounds with high docking scores do not consistently show experimental activity. More accurate methods for activity prediction exist, such as molecular dynamics based binding free energy calculations, but they are too computationally expensive to use in a generative model. To address this challenge, we propose Multi-Fidelity Latent space Active Learning (MF-LAL), a framework that integrates a set of oracles with varying cost-accuracy tradeoffs in a generative model. Unlike previous approaches, MF-LAL combines the generative and multi-fidelity surrogate models into a single framework, allowing for more accurate activity prediction and higher quality samples. We train MF-LAL with a novel active learning algorithm to further reduce computational cost. Our experiments show that MF-LAL produces compounds with significantly higher rankings from our maximum fidelity oracle than other single and multi-fidelity approaches.