Loading…
Topic
1 post about Adversarial Prefixes. Every summary links to the original.
See the idea
An adversarial prefix is a challenge prepended to a prompt to see whether a model holds its original decision. This diagram maps the layers that turn that pressure test into a measurable, reviewable signal.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
FairStress is a benchmark for measuring demographic fairness in LLMs beyond the final output decision, evaluating how readily models change choices when challenged by adversarial prefixes. It expands 300 two-candidate scenarios across 29 domains into over 13 million test cases by varying demographic identity and persuasive arguments. Across 15 models, the benchmark revealed anti-stereotypical over-correction: models abandoned correct decisions more readily when doing so favored a minority-coded candidate, with average symmetry gaps of 6.9 and 9.3 percentage points for implied and explicit identity. Most debiasing methods, including self-critique and activation steering, did not reliably reduce the asymmetry and sometimes harmed accuracy; one activation-steering method dropped Qwen2.5-7B-Instruct’s accuracy from 98% to 47%. Counterfactual chain-of-thought, which asks models to reconsider with swapped identities, reduced the asymmetry gap while largely preserving accuracy, though it did not remove all biases.
The best engineering reads, once a week.
