ai_extract: Spot Fixes vs. a Measured Eval Loop
A side-by-side comparison of two ways to improve ai_extract document extraction: patching one field at a time, or versioning prompts and judging them against fixed ground truth.
Hugging FaceKyle WiggersProblem & solution
The post presents a study by Ai2 on the reliability of LLMs as novelty judges for scientific research ideas, using six models including GPT-5.4 and Claude Opus 4.6. The researchers mined ICLR 2026 peer reviews to create a validated pool of 154 high-novelty and 145 lower-novelty human ideas, then paired them with 154 simple AI-generated ideas. Testing revealed that small prompt changes could shift accuracy by over 50 percentage points on AI-generated ideas, such as GPT-5.4 dropping from 93.5% to 40.9% with a simpler prompt. Additional search and reasoning did not consistently improve judgments; a simple Claude Opus 4.6 baseline achieved 0.854 macro-F1 at $2.59, while AI Scientist scored 0.772 at $82.05. The findings indicate that LLM-based novelty evaluation remains unreliable, particularly for AI-generated ideas, and the authors release benchmark data and an automated pipeline for future research.
The study is part of Ai2’s broader work on AI systems for scientific discovery, including projects like DataVoyager, AutoDiscovery, and Theorizer, which aim to support researchers in exploring data, generating hypotheses, and synthesizing evidence. The need for reliable novelty evaluation arises as these systems produce research ideas that must be assessed for originality.
The researchers built an automated pipeline that mines ICLR 2026 peer reviews for explicit mentions of originality, creating validated high- and low-novelty pools of human-authored ideas. They paired these with simple AI-generated ideas to form two evaluation settings (Human-Only and Human+Generated). They then tested six LLM judges (GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, Claude Opus 4.6) under varying prompts, reasoning budgets, retrieval conditions, and voting configurations to measure reliability.
Small changes to a novelty judge’s prompt can shift accuracy by over 50 percentage points, as seen when GPT-5.4 dropped from 93.5% to 40.9% on Human+Generated pairs with a simpler prompt.
Additional literature search and higher reasoning budgets do not consistently improve novelty judgments; a simple prompted Claude Opus 4.6 baseline outperformed both Idea Novelty Checker and AI Scientist while costing significantly less.
Novelty judges that appear reliable on human-authored research may behave very differently when evaluating AI-generated ideas, making validation on AI-generated content essential.
The researchers mined ICLR 2026 peer reviews for explicit discussions of originality, creating a high-novelty pool of 154 accepted papers in the top 10% with positive novelty comments and a lower-novelty pool of 145 rejected papers in the bottom 10% with originality concerns.
On Human+Generated, a simple Claude Opus 4.6 prompt achieved 0.854 macro-F1 at $2.59, while Idea Novelty Checker scored 0.642 at $30.66 and AI Scientist scored 0.772 at $82.05—showing that dedicated pipelines were both less accurate and much more expensive.
The study releases a benchmark with data, an automated pipeline for building evaluation sets, and code for testing novelty judges, enabling researchers to apply the approach to other peer review datasets and models.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
A side-by-side comparison of two ways to improve ai_extract document extraction: patching one field at a time, or versioning prompts and judging them against fixed ground truth.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.