A longstanding challenge for self-driving development is the ability to simulatedynamic driving scenarios seeded from recorded driving logs. Given an initialscene observed during a test drive, we seek the ability to sample plausible scene-consistent future trajectories for all agents in the scene, even when the actions for asubset of agents are chosen by an external source, such as a black-box self-drivingplanner. In order to model the complicated spatial and temporal interaction acrossagents in driving scenarios, we propose to tokenize the motion of dynamic agentsand use tools from language modeling to model the full sequence of multi-agentactions. Our traffic model explicitly captures intra-timestep dependence betweenagents, which is essential for simulation given only a single frame ofhistorical scene context, as well as enabling improvements when provided longerhistorical context. We demonstrate competitive results sampling scenarios giveninitializations from the Waymo Open Dataset with full autonomy and partialautonomy, highlighting the ability of our model to interact with agents outside its control.