3x Faster Search: Parallel Test-Time Scaling with Instructed-Retriever-1

Summary

Instructed-Retriever-1 is introduced as the retrieval-specialized model behind an update to Agent Bricks Knowledge Assistant, targeting search quality and latency. It parallelizes query and filter generation for broader candidate recall, then applies multi-pivot groupwise reranking to improve precision, using one model for both retrieval stages. The model was trained on synthetic enterprise-style environments and evaluated on realistic Knowledge Assistant workloads, where it scored 81.0 nDCG@10 for reranking and matched Claude Sonnet 4.5 retrieval quality on KARLBench. Serving uses a Mixture-of-Experts architecture, FP8 quantization, speculative decoding, and infrastructure tuning; the update reports more than 3x faster search, 2x faster answer generation, and TTFT around two seconds. End-to-end latency was consistently below 10 seconds in the offline evaluation setup.

Takeaways

1. 1

   Parallel query and filter generation improves candidate-set recall, while increasing the number of pivots provides a precision-oriented test-time scaling control for reranking.

2. 2

   On the internal realistic-workload evaluation, Instructed-Retriever-1 achieved 81.0 nDCG@10 for reranking, compared with 80.1 for Claude Sonnet 4.5; both improved over no reranking.

3. 3

   FP8 quantization showed no statistically significant quality degradation relative to BF16, and speculative decoding added more than 30% speed-up to the combined query-generation and reranking path.