From Preferences to Principles: Rubric-Based Alignment for Grounded Knowledge Answers

Summary

Designing reward signals for open-domain question answering remains challenging because high-quality answers must satisfy multiple distinct criteria that holistic scalar objectives fail to capture. To address this limitation, researchers developed a rubric-based reward framework that produces query-specific rubrics grounded in retrieved evidence and decomposed into specific quality dimensions. This structured reward setup provides fine-grained supervision during the post-training phase. Across the evaluation axes of composition, grounding, and instruction-following, the multi-dimensional grounded rubric method outperforms instruction-tuned baselines by 6.5% and flat rubric variants by 4%. Grounding rubrics in retrieved evidence bolsters factual support, while dimension decomposition enhances organization, coherence, and requirement adherence.

Takeaways

1. 1

   Conditioning rubrics on retrieved evidence improves factual support in open-domain question answering.

2. 2

   Decomposing rubrics into quality-specific dimensions enhances response coherence, organization, and adherence to query requirements.

3. 3

   The rubric-based framework improved performance by 6.5% over the instruction-tuned baseline and by 4% over flat rubric variants across composition, grounding, and instruction-following axes.