Large Language Models (LLMs) like GPT are often conceptualized as passive predictors, simulators, or even ’stochastic parrots’. We explore a novel conceptualization of LLMs, drawing on the theory of active inference originating in cognitive science and neuroscience. We examine similarities and differences between traditional active inference systems and LLMs, leading to the conclusion that currently LLMs lack a tight feedback loop between acting in the world and perceiving the impacts of their actions, but otherwise fit in the paradigm. We list a reasons why the loop may get closed, and possible consequences of this, including enhanced model self-awareness and the drive to minimize prediction error by changing the world.