Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading post…
AppleBenchmark
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study
Summary
Controlling the output of Large Language Models (LLMs) is a central challenge for their reliable deployment, yet current conditioning methods are often evaluated narrowly on effectiveness without considering generation quality. This study systematically investigates a range of conditioning methods in both concept injection and removal scenarios, including efficient steering, activation steering, prompting, and supervised fine-tuning. The authors find that efficient steering methods frequently achieve their conditioning goal at a steep cost to fluency, degrading generation quality. They identify a critical previously overlooked interaction: activation steering methods are far less effective on instruction-tuned models than on their base counterparts. Simple prompting and supervised fine-tuning are viable options for concept injection but are not as good at concept removal, while cheaply computed textual metrics highly correlate with costly LLM-as-judge scores, providing practical insights for evaluation.
Approach / What changed
Systematic investigation of conditioning methods including efficient steering, activation steering, prompting, and supervised fine-tuning in concept injection and removal scenarios.
Takeaways
Efficient steering methods achieve conditioning at a steep cost to fluency, degrading generation quality.
Activation steering methods are far less effective on instruction-tuned models than on base models, a previously overlooked interaction.
Simple prompting and supervised fine-tuning are viable for concept injection but not as effective for concept removal, while cheap textual metrics correlate well with costly LLM-as-judge scores.
FAQ
How do efficient steering methods affect fluency?
Efficient steering methods achieve conditioning at a steep cost to fluency, degrading generation quality compared to methods that preserve fluency.
Are activation steering methods equally effective on instruction-tuned and base models?
No, the study finds activation steering methods are far less effective on instruction-tuned models than on their base counterparts, a critical but previously overlooked interaction.
What are the best methods for concept injection and removal?
Simple prompting and supervised fine-tuning are viable options for concept injection, but they are not as good at concept removal. The study suggests different methods suit different scenarios.
Source and freshness
- Source
- machinelearning.apple.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.