Large language models present a challenge for computer scientists: they are complex enough that they demonstrate capacities that surprise even their creators, and constraints on access mean that researchers are often forced to try to understand these systems based purely on their behavior. Fortunately, this is a situation that is very familiar to cognitive scientists. In this talk I will describe some of the kinds of tasks that cognitive scientists have used to try to understand how equally complex human minds work based purely on their behavior, showing how they can be used to gain insight into large language models. I will describe algorithmic tasks, such as iterated learning and Markov chain Monte Carlo, which can be used to explore the knowledge implicit in these systems, and highlight some of the ways in which psychology can contribute to understanding when strategies such as using chain of thought prompting might backfire.