Physical networks can learn desirable functions using local learning rules in space and time. Real learning systems, like natural neural networks, can learn out of equilibrium, on timescales comparable to their physical relaxation. Here we study coupled learning, a framework that supports learning in equilibrium in diverse physical systems. Relaxing the equilibrium assumption, we study experimentally and theoretically how physical resistor networks learn allosteric functions far from equilibrium. We show how fast learning produces oscillatory dynamics beyond a critical threshold, and that learning succeeds well beyond that threshold. These findings show how coupled learning rules may train systems much faster than assumed before, suggesting their applicability to slowly relaxing physical systems.