A new generation of radio telescopes are being built to map the growth of cosmological structure throughout the majority of the observable universe, giving us access to new cosmological information that will shed light on outstanding questions in astrophysics and cosmology. These telescopes use 21\,cm emission from neutral hydrogen as a tracer of structure, but at the low radio frequencies that they operate face a daunting systematics suppression challenge. These systematics are wide ranging, and are generally considerably brighter than the underlying cosmological signal of interest, setting up a delicate signal separation problem that has yet to be overcome by the field. We present a new framework based on differentiable forward models that will enable the joint modeling of systematics in an end-to-end manner for the first time, allowing us to better subtract low-level systematics and compute more robust errorbars. This framework is made possible by high-performance machine learning frameworks that use automatic differentiation to quickly compute exact posterior gradients that are then fed to gradient-aware optimization and posterior sampling routines.