Imitation learning (IL) aims at producing agents that can imitate anybehavior given a few expert demonstrations. Yet existing approaches requiremany demonstrations and/or running (online or offline) reinforcementlearning (RL) algorithms for each new imitation task. Here we show that recent RL foundation models based on successor measures canimitate any expert behavior almost instantlywith just a few demonstrations and no need for RL or fine-tuning, whileaccommodating several ILprinciples (behavioral cloning, feature matching, reward-based, andgoal-based reductions).In ourexperiments, imitation via RL foundation models matches, and oftensurpasses, the performance of SOTA offline IL algorithms, and producesimitation policies from new demonstrations within seconds instead of hours.