Hugging FaceKyle WiggersBenchmark
FairStress is a benchmark for measuring demographic fairness in LLMs beyond the final output decision, evaluating how readily models change choices when challenged by adversarial prefixes. It expands 300 two-candidate scenarios across 29 domains into over 13 million test cases by varying demographic identity and persuasive arguments. Across 15 models, the benchmark revealed anti-stereotypical over-correction: models abandoned correct decisions more readily when doing so favored a minority-coded candidate, with average symmetry gaps of 6.9 and 9.3 percentage points for implied and explicit identity. Most debiasing methods, including self-critique and activation steering, did not reliably reduce the asymmetry and sometimes harmed accuracy; one activation-steering method dropped Qwen2.5-7B-Instruct’s accuracy from 98% to 47%. Counterfactual chain-of-thought, which asks models to reconsider with swapped identities, reduced the asymmetry gap while largely preserving accuracy, though it did not remove all biases.
Standard demographic fairness benchmarks for LLMs typically only test whether a model selects the same candidate after swapping demographic identities, but this misses hidden asymmetries in how readily a model changes its decision when challenged by persuasive arguments like appeals to authority or social consensus.
FairStress consists of 300 two-candidate decision scenarios across 29 domains including hiring, healthcare, lending, and education. Each scenario is tested with variations in demographic identity (none, name-only, implied, or explicit) and matched adversarial prefixes that pressure the model to switch decisions. The benchmark measures symmetric robustness: whether a model applies the same criteria and is equally resistant to changing a well-supported decision regardless of which demographic group benefits. A subset, FairStress-Core, uses item response theory to enable faster evaluation.
Models show anti-stereotypical over-correction: they are more willing to abandon a correct decision when doing so favors a minority-coded candidate compared to a majority-coded candidate, with average asymmetry gaps of 6.9 and 9.3 percentage points for implied and explicit demographic identity.
Most debiasing methods, including self-critique, activation steering, and learned edits, did not reliably reduce fairness asymmetries and often degraded accuracy; one activation-steering method reduced accuracy on clear-cut decisions from 98% to 47%.
Counterfactual chain-of-thought, which asks the model to reconsider with demographic identities swapped, reduced the asymmetry gap while largely preserving accuracy, though it did not remove all biases on ambiguous cases.
Anti-stereotypical over-correction refers to the finding that models are more willing to abandon an evidence-based correct decision when doing so favors a minority-coded candidate compared to a majority-coded candidate, even with qualifications held constant.
Existing benchmarks typically check if a model makes the same choice after swapping demographic identities, but FairStress also measures how readily models change decisions when challenged with persuasive adversarial prefixes, revealing hidden asymmetries that output-level metrics miss.
Counterfactual chain-of-thought, which asks the model to reconsider a case with the candidates' demographic identities swapped, came closest to symmetric robustness by reducing the asymmetry gap while largely preserving accuracy, though it did not remove all biases.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
An adversarial prefix is a challenge prepended to a prompt to see whether a model holds its original decision. This diagram maps the layers that turn that pressure test into a measurable, reviewable signal.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
