Optimizing Agent Reliability and Tool Calling Efficiency for Chip Design Using NVIDIA NeMo Gym

Summary

AI agents in chip design often produce plausible but incorrect artifacts, making reliability a key barrier to production adoption. NVIDIA NeMo Gym, an open library for evaluating models and agents using environments, solves this by diagnosing failure modes in multistage workflows through verification of final outputs against reference artifacts. Applied to the Siemens Fuse EDA AI Agent for standard-cell characterization, NeMo Gym uses a verifier that checks coverage, validity, relative validation, integrity, and accuracy of generated Liberty files against known references. After refining the agent’s skills and MCP tool descriptions based on trajectory analysis of failures, the composite benchmark score improved from approximately 55% to 95%, while agent turns fell from 155 to 53 and output tokens dropped from ~81K to 29K. This method generalizes to other engineering domains by emphasizing evaluation of final artifacts and domain-specific verifiers.

Takeaways

1. 1

   Evaluating agents on final Liberty file artifacts catches failures that intermediate checks miss, such as missing cells, incorrect thresholds, and timing values outside tolerance.

2. 2

   Refining MCP tool descriptions with explicit JSON schemas and simplifying overlapping file and shell interfaces greatly improves tool selection and reduces agent errors.

3. 3

   Using NeMo Gym’s iterative optimization, the composite verification score rose from ~55% to ~95%, agent turns dropped 66% (155 to 53), and output tokens dropped 64% (~81K to 29K).

FAQ