---
title: "Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples"
description: "DIN Deploy is an open-source collection of C++ samples combining ONNX Runtime with NVIDIA TensorRT RTX for local AI inference on Windows and Linux. It separates model conversion from deployment: Python exports models to ONNX, then native C++ runs inference via ONNX Runtime session and tensor APIs. Supported workloads include automatic speech recognition (Whisper offline, Parakeet TDT streaming), interactive segmentation (SAM 2.1), and prompt-driven image generation (FLUX.2-klein-4B). The FLUX.2 sample includes Vulkan and DirectX graphics interop and demonstrates post-training quantization with NVIDIA Model Optimizer, producing quantized ONNX models as drop-in replacements without code changes. On DGX Spark, GPU acceleration provides speedups: Whisper large-v3-turbo achieves 58.5x real-time factor, Parakeet TDT 206.41x, and SAM 2.1 runs at 38.3 FPS on GPU versus 0.5 FPS on CPU."
---

# Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Luca Spindler · Oct 1, 2026

**Type:** Explainer

## Summary

DIN Deploy is an open-source collection of C++ samples combining ONNX Runtime with NVIDIA TensorRT RTX for local AI inference on Windows and Linux. It separates model conversion from deployment: Python exports models to ONNX, then native C++ runs inference via ONNX Runtime session and tensor APIs. Supported workloads include automatic speech recognition (Whisper offline, Parakeet TDT streaming), interactive segmentation (SAM 2.1), and prompt-driven image generation (FLUX.2-klein-4B). The FLUX.2 sample includes Vulkan and DirectX graphics interop and demonstrates post-training quantization with NVIDIA Model Optimizer, producing quantized ONNX models as drop-in replacements without code changes. On DGX Spark, GPU acceleration provides speedups: Whisper large-v3-turbo achieves 58.5x real-time factor, Parakeet TDT 206.41x, and SAM 2.1 runs at 38.3 FPS on GPU versus 0.5 FPS on CPU.

## Approach / What changed

DIN Deploy provides C++ samples that use ONNX Runtime with the TensorRT RTX execution provider for hardware-accelerated local AI inference. Models are exported to ONNX from Python and run via native C++ using ORT session/tensor APIs. Vendor-specific code is isolated to optional accelerated paths. Graphics interop in FLUX.2 uses Vulkan and DirectX. Post-training quantization via NVIDIA Model Optimizer yields drop-in ONNX models.

## Takeaways

- Model conversion and deployment are fully separated: Python exports a model to ONNX, and native C++ runs it with ONNX Runtime without requiring a model-specific runtime.
- Shared inference code uses ONNX Runtime tensor APIs for portability; vendor-specific CUDA APIs appear only in optional accelerated paths.
- Post-training quantization with NVIDIA Model Optimizer produces quantized ONNX models that are drop-in replacements requiring no application code changes.

## FAQ

### Which platforms does DIN Deploy support?

DIN Deploy supports Windows and Linux with CMake presets for x86-64 and Arm64. DirectX is available only on Windows.

### How much faster is GPU inference vs CPU in these samples?

On DGX Spark, Whisper large-v3-turbo reaches 58.5x real-time factor on GPU vs 3.8x on CPU; Parakeet TDT reaches 206.41x vs 14.44x; SAM 2.1 runs at 38.3 FPS on GPU vs 0.5 FPS on CPU.

### Does quantization require changing application code?

No. Quantized ONNX models produced by NVIDIA Model Optimizer are drop-in replacements; because ONNX interfaces remain unchanged, no application code changes are needed.

**Tags:** [Local AI](https://yomu.fyi/topic/local-ai), [NVIDIA](https://yomu.fyi/topic/nvidia), [ONNX](https://yomu.fyi/topic/onnx), [TensorRT](https://yomu.fyi/topic/tensorrt)

- Source: [NVIDIA](https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples)
- Source URL: https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples
- Ingested by Yomu: 2026-10-01T18:00:12.407Z

[Read original post](https://developer.nvidia.com/blog/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples)