Loading post…
Hugging FaceJingang Qu, Valter Hudovernik, Martin Jurkovic, Cedric Lorenz, Akihiro Nitta, Dmitry Gordeev, Federico Lopez, Ramona Bendias, Gilberto Titericz Jr, Aleksandar S. Sokolovski, Isabel Hulseman, Jure Leskovec, Matthias FeyAnnouncement
NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction
Summary
NVIDIA Kumo Tabular is an open foundation model for tabular classification and regression that predicts labels in a single forward pass without training, tuning, or feature engineering. It uses a Transformer architecture with column, row, and in-context attention, including length-aware attention temperature to maintain sharp attention on large tables. Pretrained entirely on artificial tables generated from structural causal models with realistic imperfections such as missing values, coarsened features, and heavy-tailed targets, it comes in three sizes (28M to 215M parameters) and handles missing values without imputation. On the TabArena, BeyondArena, TALENT, and ScoringBench benchmarks, Kumo Tabular achieves top rankings and establishes a new accuracy-efficiency frontier, running 17x faster than LimiX-2. The model is available on Hugging Face under the OpenMDW-1.1 license and runs via NVIDIA’s open-source structured-data-models library.
Context
Traditional tabular machine learning requires collecting labels, engineering features, and training a new model for each task. Kumo Tabular is a foundation model that predicts labels in a single forward pass using in-context learning, eliminating the need for task-specific training.
Approach / What changed
Kumo Tabular uses a Transformer architecture with column attention to understand column distributions, row attention to learn feature interactions, and in-context attention for prediction. It employs Fourier features for cell embeddings, length-aware attention temperature to maintain sharpness on large tables, and test-GQA for efficient inference. The model is pretrained on artificial tables from structural causal models with realistic imperfections such as missing values, coarsened features, and heavy-tailed targets.
Takeaways
Kumo Tabular achieves top rankings on four tabular benchmarks (TabArena, BeyondArena, TALENT, ScoringBench) while running 17x faster than LimiX-2.
The model requires no training, tuning, or feature engineering; it predicts labels in a single forward pass from a labeled context table.
Kumo Tabular handles missing values without imputation and supports up to 10 classes natively, with error-correcting output codes for more classes.
FAQ
How does Kumo Tabular handle missing values?
Missing values are treated specially without imputation. The model was pretrained on artificial tables with various missingness patterns, so it learns to handle them directly.
What hardware is needed to run Kumo Tabular?
The post evaluates Kumo Tabular on a single RTX 6000 Pro GPU. The library is GPU-native and runs on CUDA.
What types of data does Kumo Tabular support?
It works on numerical and categorical columns only. Text, images, or timestamps can be turned into features via built-in preprocessing recipes.
Source and freshness
- Source
- huggingface.co
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.