Current active learning strategies for vision transformers in semantic segmentation require extensive pixel labeling, impractical when only one pixel per image is labeled. TEAL is the first pixel-level active learning framework built around ViT representations, using a frozen backbone and diversity-based selection directly on the native token lattice to avoid interpolation artifacts. It applies a two-level MaxHerding strategy over multi-layer token descriptors for representative candidate selection, followed by margin-based uncertainty refinement on a decoder grid. Across CamVid, Cityscapes, ADE20K, and Pascal Context, TEAL improves over the strongest baseline by up to +24.36 mIoU with a single labeled pixel per image and reaches 97% of fully supervised performance with just 10 pixels. The improvements persist when all baselines use the same DINOv3 backbone, showing that token-space selection with decoder-grid refinement makes ViT token geometry effective for low-budget active segmentation.
Context
Research paper introducing TEAL, a token-space active learning framework for low-budget semantic segmentation using frozen ViT backbones.
Approach / What changed
TEAL employs a frozen ViT backbone, diversity-based selection on the token lattice via two-level MaxHerding over multi-layer token descriptors, and margin-based uncertainty refinement on a decoder grid.
Takeaways
1
TEAL operates directly on the ViT token lattice, avoiding interpolation artifacts from converting token embeddings to dense pixel resolutions.
2
With a single labeled pixel per image, TEAL improves over the strongest baseline by up to +24.36 mIoU on standard segmentation datasets.
3
TEAL reaches 97% of fully supervised performance with just 10 labeled pixels per image, demonstrating extreme label efficiency.
FAQ
How does TEAL select informative pixels?
TEAL uses a two-level MaxHerding strategy over multi-layer token descriptors for diversity-based selection, followed by margin-based uncertainty refinement on a finer decoder grid.
What datasets were used to evaluate TEAL?
TEAL was evaluated on CamVid, Cityscapes, ADE20K, and Pascal Context, consistently outperforming baselines under extreme label scarcity.
Does TEAL require a fine-tuned ViT backbone?
No, TEAL uses a frozen ViT backbone, avoiding the need for fine-tuning the encoder while still achieving strong performance.
Active learning is a loop that spends a fixed labeling budget on the samples most worth annotating. This diagram maps its stages — pool, score, label, train, evaluate — and the return path that closes the cycle.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.