Large language models (LLMs) have demonstrated impressive capabilities in text generation, but their ability to reason over complex data remains an area of ongoing research. In this talk, we present three distinct approaches to improve LLM reasoning over complex structures.

First, we leverage graph algorithms to analyze and understand the reasoning capabilities of transformer models. Our results establish a representational hierarchy, revealing the necessary Transformer capacity (number of layers, embedding dimension size) for solving different classes of reasoning tasks.

Next, we exploit the topology of temporal reasoning to generate novel synthetic problem instances. This allows for a more robust evaluation of LLM reasoning capabilities.

Finally, we introduce a method for improving in-context representations of structured data for pretrained LLMs, facilitating more effective reasoning over complex information.