Poster
in
Workshop: Table Representation Learning Workshop
Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks
Benjamin Feuer · Niv Cohen · Chinmay Hegde
Keywords: [ Deep Learning ] [ Benchmarking ] [ Representation Learning ] [ tabular ] [ gradient boosted decision trees ]
Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted Networks such as TabPFN have successfully learned to classify tabular data in-context: the model parameters are designed to classify new samples based on labelled training samples given after the model training. While such models show great promise, their applicability to real-world data remains limited due to the computational scale needed. We conduct an initial investigation of sketching and feature-selection methods for TabPFN, and note certain key differences between it and conventionally fitted tabular models.