I will summarize research in the area of intrinsic motivation in the context of learning and exploration and touch upon open-ended learning in the IMOL community. I will then present our recent work on combining different intrinsic motivation signals with reinforcement learning, such as learning progress, causal influence and information gain. A particular exciting direction is to employ model-based reinforcement learning to make robots learn by freely playing how to interact effectively driven by information gain and other generic drives. We find that this leads to high zero-shot generalization to new tasks.