Information theory has a long track record as a widely adopted framework to study complex systems. This remains true within the context of cognitive systems, where it has been utilized in psychology, neuroscience, artificial intelligence, artificial life etc. A crucial aspect to what makes a system cognitive is the fact that they are in continuous closed-loop interaction with their environments. This brings certain challenges to utilizing information theory in this context: these systems are multivariate, information has temporal dynamics, information could be distributed across the system and finally, cognition doesn't have to be limited to the brain. In this article, we provide perspectives on these challenges, explain their significance, provide examples of how they have been tackled in other work, and outline the open challenges.