# An Engineer’s Guide to Better AI Skills: Implementing a Testing Process to Optimize Agent…

[Pinterest](https://yomu.fyi/company/pinterest) · Pinterest Engineering · May 12, 2026

**Type:** Problem & solution

## Summary

Software engineers deploying AI agents often encounter unreliable skill invocation for repository-specific architectures, such as Pinterest's iOS framework. To quantify and resolve this issue, a Bash test harness piped positive and negative prompts into Pin-agent and Claude Code while parsing streamed JSON logs for execution signatures. Initial vanilla benchmarks yielded baseline accuracies of 73% for GPT 5.2-codex and 62% for Opus 4.5 Claude. Optimizations such as expanding YAML frontmatter descriptions with architectural context, utilizing capitalized imperative directives, and adding reference tables in AGENTS.md raised invocation rates. Both agents reliably loaded skills when prompts contained explicit invocations, demonstrating that empirical testing paired with descriptive prompting stabilizes agent skill adoption.

## Context

AI agents failed to consistently load a domain-specific knowledge skill (rx-mvvm) for Pinterest's iOS architecture, particularly during architectural reviews when engineers supplied terse or ambiguous prompts.

## Approach / What changed

Built a Bash test harness that pipes 15 positive and 5 negative prompt cases across multiple runs into Pin-agent and Claude Code, parses JSON logs for skill invocation patterns, and measures accuracy across optimizations like frontmatter descriptions, aggressive language, and AGENTS.md tables.

## Takeaways

- Baseline skill invocation accuracy was 73% on Pin-agent (GPT 5.2-codex) and 62% on Claude Code (Opus 4.5) when subjected to terse test prompts.
- Adding contextual architectural information to skill YAML frontmatter improved skill invocation rates across both tested agents.
- Every test run on both agents successfully loaded the skill when the prompt explicitly instructed the agent to load it.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Developer Experience](https://yomu.fyi/topic/developer-experience), [iOS](https://yomu.fyi/topic/ios), [LLMs](https://yomu.fyi/topic/llm), [Testing](https://yomu.fyi/topic/testing)

[Read original post](https://medium.com/pinterest-engineering/an-engineers-guide-to-better-ai-skills-implementing-a-testing-process-to-optimize-agent-a000c9c9abcd)