While meta-learning algorithms are often viewed as algorithms that learn to learn, an alternative viewpoint frames meta-learning as inferring a hidden task variable from experience consisting of observations and rewards. From this perspective, learning-to-learn is learning-to-infer. This viewpoint can be useful in solving problems in meta-reinforcement learning, which I’ll demonstrate through two examples: (1) enabling off-policy meta-learning and (2) performing efficient meta-reinforcement learning from image observations. Finally, I’ll discuss how I think this perspective can inform future meta-reinforcement learning research.