Loading…
Topic
Sentence Transformers
1 post about Sentence Transformers. Every summary links to the original.
- 1 post
- 1 company
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Token Pooling-Aware Training for Multi-Vector Embedding Models
Multi-vector embedding models store many token vectors per document, causing large indexes. Token pooling via Ward hierarchical clustering merges similar token embeddings into groups, reducing the number of vectors, but aggressive compression can lose retrieval-relevant distinctions. Pooling-aware training addresses this by averaging a retrieval loss on unpooled and pooled embeddings, using a straight-through estimator to pass gradients through the discrete clustering. At pool_factor=4, a normally trained model retains 94.6% of its unpooled nDCG@10, while pooling-aware training raises retention to 97.8% at the same 4x reduction in document vectors. Adaptive pooling with a calibrated threshold also benefits, retaining 97.3% of the unpooled score while storing 84.7% fewer vectors.
Weekly digest
The best engineering reads, once a week.