Speakers are thought to use rational information transmission strategies for efficient communication; for example, they keep the information density of their sentences uniform over the course of written texts (Genzel and Charniak, 2003; 2003)—especially so within coherent contextual units, such as paragraphs. In this work, we test whether, and within which contextual units, speakers adhere to the principle of uniform information density (Jaeger and Levy, 2007) in written monologue as well as in written and spoken task-oriented dialogue. Using a pre-trained Transformer-based language model, which provides more robust measurements than the n-gram models used in prior work, we confirm that speakers adhere to the principle in newspaper articles and present new evidence that they also do in written cooperative reference games as well as in spoken dialogues involving instruction giving and following. Because patterns of information transmission vary within different contextual units, we then use the context window of our language model to estimate information density as a function of the relevant utterance context; this was never explicitly measured in previous related work. We find that, when context is explicitly factored in, speakers transmit information at a stable rate in newspaper articles but that this rate decreases in spoken open domain and written task-oriented dialogues. We suggest that a more faithful model of communication should include production efforts and goal-oriented rewards. Our hope is that this line of work will inform the development of dialogue generation models that organise the transmission of information in a more human-like fashion.