TabPFN and TabICL Beat Tuned XGBoost on All 14 Tables Without Training

TabPFN and TabICL vs. tuned XGBoost: the model that doesn't train won 14/14

A hands-on benchmark pits two pretrained tabular foundation models, TabPFN and TabICL, against tuned XGBoost across fourteen datasets. The models that never see the training data during fitting win on all fourteen, often by small but consistent margins, and answer in about a second. Tuning XGBoost took up to fifty seconds per dataset, suggesting hyperparameter search may become optional for many tabular tasks.