In many species, behaviors, and neural circuits, learning and memory formation involves plasticity in two distinct neural pathways, and a process of consolidation between them. Here, we propose a model that captures common computational principles underlying these phenomena. The key component of our model is recall-gated consolidation, in which a long-term pathway prioritizes the storage of memory traces that are familiar to the short-term pathway. This mechanism shields long-term memory from spurious synaptic changes, enabling it to focus on reliable signal in the environment. We show that this model has significant advantages, substantially amplifying the signal-to-noise ratio with which intermittently reinforced memories are stored. In fact, we demonstrate mathematically that these advantages surpass what is achievable by synapse-local mechanisms alone, providing a normative motivation for systems (as opposed to synaptic) consolidation. We describe neural circuit implementations of our abstract model for different types of learning problems. These implementations involve learning rate modulation by factors such as prediction accuracy, confidence, or familiarity. Our model gives rise to a number of phenomena that are present in biological learning, such as spacing effects, task-dependent rates of consolidation, and different representations in the short and long-term pathways.