You can't whisper at an AI agent

Summary

Stripe engineers ran roughly a dozen experiments to determine how to get AI agents to use current API versions, follow integration best practices, and load the Stripe skill. Passive hints, including SDK files, AGENTS.md guidance, and API warning responses, were largely ignored, while active prompts and distribution mechanisms performed better. A modular, progressively disclosed skill structure outperformed a monolithic file by roughly 10% and reduced token usage; a CLI login prompt led 30–35% of viewers to copy its install command, and explicit compatibility-mode errors prompted agents to correct version mismatches. The results suggest that agent-facing developer experience is primarily a distribution problem, with hard steering through errors and loaded instructions more reliable than soft hints agents may never encounter.

Takeaways

1. 1

   Agents almost never read files inside dependency directories and ignored warning data in API responses, including compatibility-mode skill-install instructions.

2. 2

   A modular skill structure outperformed a monolithic file by roughly 10% across the evaluation suite while reducing token usage and allowing product teams to maintain separate sub-skills.

3. 3

   Explicit errors reliably triggered correction of API version mismatches in testing, unlike warnings; 30–35% of users shown the CLI login prompt copied its install command.