Loading…
Topic
Tabular Foundation Models
2 posts about Tabular Foundation Models. Every summary links to the original.
- 2 posts
- 2 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Tabular foundation models on Databricks AI Runtime
Tabular foundation models (TFMs) are pretrained transformers that make predictions on new tabular datasets without task-specific training, using in-context learning on labeled rows passed as context. They eliminate the need for bespoke model development and maintenance pipelines by inferring relationships from millions of diverse synthetic datasets used during pretraining. However, TFMs require GPU infrastructure because their transformer architecture and in-context learning demand dense matrix multiplications and large memory for activations, which grows with dataset size. Databricks AI Runtime addresses this by providing managed serverless GPU infrastructure directly where data resides, with integrated observability and governance through Unity Catalog. The platform supports zero-shot inference and fine-tuning of models like KumoTabular, enabling enterprises to deploy TFMs as batch pipelines or real-time endpoints without creating data silos.
Mitra-v2: Scaling synthetic pretraining for tabular foundation models
Mitra-v2 is a second-generation tabular foundation model for classification and regression, pretrained entirely on synthetic tables. It uses a Transformer that attends across both rows and columns, and is pretrained on tables with 10 times more rows and roughly three times more columns than its predecessor. A new hybrid SCM prior combines neural networks, decision trees, Gaussian processes, and autoregressive processes within a single causal graph. On the TabArena benchmark of 51 real-world datasets, Mitra-v2 leads the single-model field and completes all datasets within the one-hour budget, which Mitra-v1 failed to do on more than a third. The model is released under Apache-2.0 on Hugging Face, with fine-tuned versions and inference code.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.