# Build On-Device AI Companions with the NVIDIA ACE Game Agent SDK and Unreal Engine 5 Plugins

[NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog) · Phillip Singh · Jun 16, 2026

**Type:** Announcement

## Summary

Game developers integrating on-device artificial intelligence companions face challenges such as conversational latency, state synchronization, and loop prevention. NVIDIA announced the open-source ACE Game Agent SDK alongside a suite of Unreal Engine 5 plugins to streamline native, hardware-accelerated character pipelines. The SDK offers Agent, Chat, and RAG APIs, enabling characters to execute multi-step tool-assisted reasoning and query game databases. Unreal Engine 5 plugins introduce local runtime models including nemo-conformer-ctc-120m for speech recognition, Qwen 3.5 4B for language generation, and Chatterbox Turbo 350M for speech synthesis. Additional tooling expands to motion generation via Animotive Kimodo and rendering updates with the DLSS 4.5 Unreal Engine plugin.

## Context

Building on-device AI non-player characters requires overcoming conversational latency, synchronizing global and dynamic game states, avoiding infinite loops, and mitigating cloud compute costs.

## Approach / What changed

NVIDIA released the open-source ACE Game Agent SDK in beta along with native Unreal Engine 5 plugins for automatic speech recognition, small language models with GGUF support, and text-to-speech, running locally on RTX hardware.

## Takeaways

- The NVIDIA ACE Game Agent SDK provides Agent, Chat, and RAG APIs for stateful reasoning, direct inference control, and hybrid knowledge retrieval.
- NVIDIA ACE Unreal Engine 5 plugins include local runtime AI models such as nemo-conformer-ctc-120m ASR, Qwen 3.5 4B SLM, and Chatterbox Turbo 350M TTS.
- The DLSS 4.5 Unreal Engine plugin integrates Dynamic Multi Frame Generation, Multi Frame Generation 6X Mode, and a second-resolution transformer for Super Resolution.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://developer.nvidia.com/blog/build-on-device-ai-companions-with-the-nvidia-ace-game-agent-sdk-and-unreal-engine-5-plugins)