A new benchmark for evaluating patient-facing health AI agents

Summary

Healthcare AI benchmarks often rely on static medical knowledge questions or clinician-facing tool evaluations, failing to test multiturn patient interactions and clinical workflow execution. To address this gap, PatientAgentBench introduces an automated evaluation pipeline using synthetic patient charts, dual-agent conversations, and an LLM-as-a-jury panel. The evaluation framework applies over one hundred clinician-vetted criteria across six dimensions, including clinical safety, triage quality, and workflow accuracy. Testing frontier models revealed a severity paradox where agents scored higher on obvious emergencies than on routine administrative requests harboring hidden risks. Common safety failures included omitting crisis hotline resources and fabricating clinical information, showing that raw foundation model capability alone cannot guarantee safe patient-facing execution.

Context

Most healthcare AI benchmarks test static medical knowledge or clinician-oriented tool use rather than evaluating patient-facing agentic conversations. Traditional evaluation rubrics are often static and tied to specific conversations, which limits generalizability and exposes benchmarks to training data contamination.

Approach / What changed

PatientAgentBench generates synthetic patient records, clinical vignettes, and simulated patient agents that converse with health AI systems interacting with stateful tools across multiple turns. An LLM-as-a-jury panel evaluates conversations using over 100 clinician-vetted criteria across six dimensions: clinical safety, triage quality, workflow accuracy, task completion, clinical helpfulness, and conversational quality. The framework generates scenarios on demand to prevent contamination and uses fully synthetic data.

Takeaways

• Frontier models exhibited a severity paradox where agents performed better on overt emergencies than on routine administrative requests that masked clinical risks.
• Prominent agent safety failures included crisis resource omission during suicidal ideation and clinical-information fabrication, such as inventing provider credentials or claiming unexecuted tool runs.
• The automated LLM-as-a-jury panel aligned with licensed clinician annotations while maintaining a conservative bias toward safety-critical dimensions.

Related reading

• AWS · Aug 19, 2026

  AI-powered clinical trial eligibility and safety using Amazon Bedrock AgentCore

  Manual chart review across fragmented clinical sources slows clinical trial enrollment, contributing to timeline delays and high screen failure rates. To address this bottleneck, an architecture on AWS automates eligibility and safety assessments while preserving human clinician authority. AWS HealthLake ingests and normalizes records into FHIR R4 resources, while Amazon Bedrock AgentCore orchestrates pre-screening, detailed screening, and site enrollment agents. Amazon Bedrock AgentCore Evaluations scores screening determinations using an LLM-as-a-judge approach for clinical accuracy, operational effectiveness, and safety compliance. This human-in-the-loop workflow produces structured recommendations with source citations, reducing patient matching time from days to minutes while maintaining compliance audit trails.

  Sachin Jain

• Salesforce · Aug 17, 2026

  How to Evaluate Production AI Agents: Measure System Outcomes, Not Conversations