Better, faster, less wrong: Enhancing issue grouping

Summary

Sentry upgraded its AI-driven issue grouping system to better prevent duplicate issues without merging distinct application errors. The team trained lightonai/modernbert-embed-large using Matryoshka Representation Learning on hundreds of thousands of stacktrace pairs labeled by Claude Sonnet 4.5. To optimize the high-throughput ingestion pipeline, embeddings were truncated from 768 to 64 dimensions, combined with bfloat16 precision, PyTorch SDPA, and CUDA graph compilation. A phased live rollout used threshold-gated index backfilling and automatic fallback to prevent issue spikes during migration. In production, the v2 model cuts the overgrouping rate from 8% to 4%, increases prevented duplicate issues to 70%, and delivers 6x faster inference.

Context

Sentry groups incoming application errors into issues using structural fingerprinting, falling back to an ML embedding model when lexical matching fails. The v1 model exhibited an 8% aggregate overgrouping rate—with specific platform overgrouping rates reaching 30% to 60%—which incorrectly merged distinct root causes into single issues and hid distinct application bugs.

Approach / What changed

Sentry built and deployed a v2 grouping model based on lightonai/modernbert-embed-large, trained on hundreds of thousands of stacktrace pairs labeled by Claude Sonnet 4.5. The model leverages Matryoshka Representation Learning, allowing embeddings to be truncated from 768 to 64 dimensions. For inference, they adopted bfloat16, PyTorch SDPA, and PyTorch model compilation. They implemented a zero-downtime migration strategy using model fallback and a training_mode parameter in Seer to backfill embeddings dynamically from live traffic.

Takeaways

• Truncating Matryoshka-trained embeddings from 768 to 64 dimensions traded a 2% accuracy loss for a 4x reduction in p50 HNSW lookup times, a 20x reduction in p50 insertion latency, and a 12x reduction in pgvector storage.
• Compiling the PyTorch model alongside bfloat16 and scaled dot product attention (SDPA) yielded nearly a 3x inference latency reduction and eliminated production CUDA out-of-memory errors.
• To migrate stateful embeddings without downtime or multi-month offline backfills, Sentry used a dual-model query fallback with a training_mode API parameter that bypasses near-duplicate suppression to backfill the v2 index on live traffic.

Related reading

• Sentry · Apr 29, 2026

  Two commands to Sentry: now on Stripe Projects

  Sentry is now integrated into Stripe Projects, enabling developers and coding agents to discover, provision, and manage Sentry services directly through the command-line interface. Running initialization and addition commands automatically sets up accounts, generates necessary environment variables like auth tokens and DSNs, and scaffolds agent skill files for tools such as Claude Code and Cursor. The integration supports plan upgrades, downgrades, and the addition of the Seer AI debugging assistant with billing handled through existing Stripe payment methods. Additional CLI capabilities include minting single-use magic login URLs for the Sentry dashboard, managing multi-member team collaboration tied to Stripe organizations, and rotating project credentials directly into local environment files.

  Burak Yiğit Kaya

• Sentry · May 21, 2026

  The product analytics you already have