In this presentation, we explore the challenges and opportunities of applying sequential modeling to tabular data in financial services. We discuss the unique characteristics of this type of data, including its heterogeneous schemas, mixed data types, and temporal nature. We then review current research in the field, highlighting the limitations of existing approaches. We propose a novel data centric approach to transforming tabular data into tokens that enables simple, interpretable techniques for data mining and leverages the success of transformers in large language models to build generalizable pre-trained models for tabular data. We demonstrate promising results on open-source datasets and conclude by discussing future research directions, including new encoding methods for numerical features, neural point process approaches, and more tokenization methods. Our work aims to contribute to the development of more effective and generalizable models for understanding behavior in financial services and other industries.