---
title: "TensorRT"
description: "2 posts about TensorRT, summarised, each linking to the original."
---

# TensorRT
> 2 posts about TensorRT, summarised, each linking to the original.

## Articles

### [Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples](https://yomu.fyi/post/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Luca Spindler
- Published: Oct 1, 2026
- Type: Explainer

DIN Deploy is an open-source collection of C++ samples combining ONNX Runtime with NVIDIA TensorRT RTX for local AI inference on Windows and Linux. It separates model conversion from deployment: Python exports models to ONNX, then native C++ runs inference via ONNX Runtime session and tensor APIs. Supported workloads include automatic speech recognition (Whisper offline, Parakeet TDT streaming), interactive segmentation (SAM 2.1), and prompt-driven image generation (FLUX.2-klein-4B). The FLUX.2 sample includes Vulkan and DirectX graphics interop and demonstrates post-training quantization with NVIDIA Model Optimizer, producing quantized ONNX models as drop-in replacements without code changes. On DGX Spark, GPU acceleration provides speedups: Whisper large-v3-turbo achieves 58.5x real-time factor, Parakeet TDT 206.41x, and SAM 2.1 runs at 38.3 FPS on GPU versus 0.5 FPS on CPU.


### [Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton](https://yomu.fyi/post/simplifying-model-serving-across-multiple-gpus-with-nvidia-tensorrt-mu.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 21, 2026
- Type: Announcement

NVIDIA TensorRT multi-device inference enables a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives, supported from TensorRT 11.0. Integrated into Dynamo-Triton release 26.07, one model instance can own multiple GPUs and expose a single gRPC endpoint so the client does not coordinate GPU ranks. The post demonstrates this with Cosmos 3 Nano video generation, where the transformer uses Ulysses context parallelism to distribute 44,160 video tokens across up to eight GPUs. End-to-end latency drops from 156.6 seconds on one GPU to 34.2 seconds on eight GPUs, achieving a 6.09x transformer RPC speedup. Teams can trade additional GPU resources for shorter request latency while keeping the application interface and surrounding workflow stable.