Funding better evaluations of AI’s impact on wellbeing

Summary

A $5 million grant program will fund independent research into how AI affects user wellbeing, offering direct funding, model access, and technical support for open-source evaluations. The initiative addresses the difficulty of judging wellbeing-related behavior, where risks may emerge only across long, shifting conversations and where otherwise reasonable advice can become harmful in a different context, such as disordered eating. Grantees will work independently and publish reusable evaluations or benchmarks, with guidance asking projects to define pass/fail criteria, involve clinical and subject-matter experts, test overcompliance and overrefusal, model realistic multi-turn scenarios, and validate graders against experts. Applications are due September 21, with selected full-proposal applicants notified by October 5.

Takeaways

1. 1

   The grant program will provide $5 million in funding for independent research into AI’s effects on user wellbeing.

2. 2

   Wellbeing evaluations should account for multi-turn conversations in which risk escalates and context changes, rather than judging isolated responses only.

3. 3

   The guidance calls for testing both overcompliance and overrefusal and validating evaluation graders against subject-matter experts.