---
title: "How UK AISI and EvalEval Are Making Benchmark Results Reproducible"
description: "The UK AI Security Institute (AISI) has begun using EvalEval’s Every Eval Ever (EEE) schema and Evaluation Cards platform to openly share LLM evaluation results. This collaboration, stemming from a joint workshop at NeurIPS 2025, aims to make benchmark results more reproducible by providing standardized configuration and context alongside verified scores. AISI released results for five benchmarks—HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0—across six frontier models, plus two cyber evaluations using a partially overlapping model set. The release accompanies a paper studying how inference-time compute and evaluation protocol affect performance, revealing that models continued solving tasks when given correctness feedback. By adopting a shared reporting schema, evaluators can compare findings across studies and understand how setup choices influence reported performance, while EvalEval invites model developers, evaluation creators, and researchers to contribute to the shared infrastructure."
---

# How UK AISI and EvalEval Are Making Benchmark Results Reproducible

[Hugging Face](https://yomu.fyi/company/hugging-face) · Avijit Ghosh, Jenny Chim, Deep Joshi, Srishti, Matt Kennedy, Irene Solaiman, Jessica McFadyen, Lynn Tan, Coz · Sep 22, 2026

**Type:** Announcement

## Summary

The UK AI Security Institute (AISI) has begun using EvalEval’s Every Eval Ever (EEE) schema and Evaluation Cards platform to openly share LLM evaluation results. This collaboration, stemming from a joint workshop at NeurIPS 2025, aims to make benchmark results more reproducible by providing standardized configuration and context alongside verified scores. AISI released results for five benchmarks—HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0—across six frontier models, plus two cyber evaluations using a partially overlapping model set. The release accompanies a paper studying how inference-time compute and evaluation protocol affect performance, revealing that models continued solving tasks when given correctness feedback. By adopting a shared reporting schema, evaluators can compare findings across studies and understand how setup choices influence reported performance, while EvalEval invites model developers, evaluation creators, and researchers to contribute to the shared infrastructure.

## Context

The reproducibility of AI evaluation results is hindered by inconsistent reporting formats and insufficient setup information. Running costly evaluations again is often prohibitive. EvalEval and AISI collaborate to address this through shared infrastructure for standardized evaluation reporting.

## Approach / What changed

AISI adopted EvalEval’s Every Eval Ever schema and Evaluation Cards platform to publicly share verified evaluation results with standardized configuration and context information, including data for five benchmarks and six frontier models.

## Takeaways

- AISI released verified evaluation results for HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0 covering Claude Opus 4, 4.5, 4.6, GPT-5, 5.2, and 5.4 models.
- The paper shows that when models received correctness feedback from an oracle after each attempt, they continued to solve additional tasks as token use increased.
- EvalEval invites model developers, evaluation developers, and researchers to report results or use the Every Eval Ever schema and Evaluation Cards to standardize evaluation reporting.

## FAQ

### What benchmarks and models are included in AISI's evaluation release?

The release includes HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0, covering Claude Opus 4, 4.5, 4.6, GPT-5, 5.2, and 5.4 models.

### How does performance on Humanity's Last Exam change with inference compute?

When models received correctness feedback from an oracle after each attempt, they continued to solve additional tasks as token use increased, showing that protocol and compute affect performance.

### Who is invited to contribute to the shared evaluation mission?

Model developers can report verified results, evaluation developers can use the Every Eval Ever schema, and researchers can explore Evaluation Cards to examine the state of evaluation reporting.

**Tags:** [AI](https://yomu.fyi/topic/ai), [Benchmark](https://yomu.fyi/topic/benchmark), [LLMs](https://yomu.fyi/topic/llm), [Model Evaluation](https://yomu.fyi/topic/model-evaluation)

- Source: [Hugging Face](https://huggingface.co/blog/evaleval-aisi)
- Source URL: https://huggingface.co/blog/evaleval-aisi
- Ingested by Yomu: 2026-09-22T18:00:09.771Z

[Read original post](https://huggingface.co/blog/evaleval-aisi)