---
title: "Jetson"
description: "2 posts about Jetson, summarised, each linking to the original."
---

# Jetson
> 2 posts about Jetson, summarised, each linking to the original.

## Articles

### [TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor](https://yomu.fyi/post/tensorrt-edge-llm-completes-the-mlperf-edge-agentic-benchmark-6-4x-fas.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 16, 2026
- Type: Benchmark

Edge agentic workloads require repeated tool-calling turns, growing conversation histories, and valid function calls on constrained devices rather than single-prompt responses. On one NVIDIA Jetson AGX Thor Developer Kit, TensorRT Edge-LLM ran Qwen3.6-27B in SingleStream mode and completed the benchmark’s 1,007-turn performance workload in 24 minutes and 36 seconds, versus 2 hours and 37 minutes for the llama.cpp reference run. It reported 52.33 tokens per second, 247.12 ms median time to first token, and 87.94% BFCL overall accuracy. The implementation combines NVFP4 weights and activations, FP8 KV cache, reusable prompt prefixes with recurrent-state restoration, and tree-based multi-token prediction; roughly 96% of prompt tokens used hot cache, while tree-based MTP could achieve about 40% more decoding performance than linear MTP with three draft steps.


### [Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson](https://yomu.fyi/post/frontier-reasoning-reaches-the-edge-how-to-deploy-and-optimize-models.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 4, 2026
- Type: Tutorial

Reasoning and agentic models have typically required data-center inference, creating network dependencies, higher costs, and potential data exposure for edge applications. The post explains how to choose between dense Qwen3.8-27B, which activates all 27 billion parameters, and mixture-of-experts Nemotron 3.5 Lightning, which has 30 billion total parameters but activates 3 billion per token. Deployment uses vLLM on Jetson with NVFP4 quantization and speculative decoding, where a draft model proposes tokens and the main model verifies them; the examples include DSpark for Nemotron and DFlash2 for Qwen. Reported tests show up to 6.28x decode-throughput speedup over BF16, while workload-specific throughput still varies. Validation with representative prompts is recommended before deployment, and custom quantization or speculator training is reserved for cases where accuracy or acceptance rates reveal a clear gap.