Abstract: We present a generative agent for stoichiometry-constrained isomer search. Our approach trains entirely in $3$D using a purely online \ac{rl} framework. Unlike prior approaches, which overfit to specific chemical formulas, we introduce a multi-composition training framework that enables the agent to generalize across a wide range of formulas. This is achieved by leveraging a reference dataset to procedurally define new generation tasks, simultaneously facilitating a more formal evaluation of the agent's discovery capabilities. Combined with new energy- and validity-based rewards, we demonstrate that our approach significantly outperforms previous work, discovering an order of magnitude more valid isomers for unseen test formulas. By addressing these challenges, we aim to reinvigorate progress in self-guided 3D molecular discovery, providing a more robust framework for future innovations in the field.