---
title: "Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson"
description: "Reasoning and agentic models have typically required data-center inference, creating network dependencies, higher costs, and potential data exposure for edge applications. The post explains how to choose between dense Qwen3.8-27B, which activates all 27 billion parameters, and mixture-of-experts Nemotron 3.5 Lightning, which has 30 billion total parameters but activates 3 billion per token. Deployment uses vLLM on Jetson with NVFP4 quantization and speculative decoding, where a draft model proposes tokens and the main model verifies them; the examples include DSpark for Nemotron and DFlash2 for Qwen. Reported tests show up to 6.28x decode-throughput speedup over BF16, while workload-specific throughput still varies. Validation with representative prompts is recommended before deployment, and custom quantization or speculator training is reserved for cases where accuracy or acceptance rates reveal a clear gap."
---

# Frontier Reasoning Reaches the Edge: How to Deploy and Optimize Models on NVIDIA Jetson

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Elizabeth Goodman · Sep 4, 2026

**Type:** Tutorial

## Summary

Reasoning and agentic models have typically required data-center inference, creating network dependencies, higher costs, and potential data exposure for edge applications. The post explains how to choose between dense Qwen3.8-27B, which activates all 27 billion parameters, and mixture-of-experts Nemotron 3.5 Lightning, which has 30 billion total parameters but activates 3 billion per token. Deployment uses vLLM on Jetson with NVFP4 quantization and speculative decoding, where a draft model proposes tokens and the main model verifies them; the examples include DSpark for Nemotron and DFlash2 for Qwen. Reported tests show up to 6.28x decode-throughput speedup over BF16, while workload-specific throughput still varies. Validation with representative prompts is recommended before deployment, and custom quantization or speculator training is reserved for cases where accuracy or acceptance rates reveal a clear gap.

## Context

Reasoning and agentic workloads have been difficult to run locally on edge hardware because capable models were too large, forcing inference through data centers and introducing network dependency, cost, and potential data-exposure concerns.

## Approach / What changed

The post uses Nemotron 3.5 Lightning and Qwen3.8-27B as deployment examples, comparing their architectures and serving them with vLLM on NVIDIA Jetson. It combines NVFP4 quantization with speculative decoding, then recommends validating model and draft-checkpoint choices using representative application workloads.

## Takeaways

- Qwen3.8-27B is a dense model that activates all 27 billion parameters per token, while Nemotron 3.5 Lightning uses a mixture-of-experts architecture with 30 billion total parameters and 3 billion active per token.
- NVFP4 reduces per-pass computation and memory use, while speculative decoding uses a draft model to propose tokens for verification; together, the tested configuration reached up to 6.28x decode-throughput speedup over BF16.
- The fastest tested speculative configuration differed by model: Nemotron 3.5 Lightning used DSpark, while Qwen3.8-27B used DFlash2; representative prompts should be used to validate throughput and acceptance rates.

**Tags:** [Jetson](https://yomu.fyi/topic/jetson), [Quantization](https://yomu.fyi/topic/quantization), [Speculative Decoding](https://yomu.fyi/topic/speculative-decoding), [vLLM](https://yomu.fyi/topic/vllm)

- Source: [NVIDIA](https://developer.nvidia.com/blog/frontier-reasoning-reaches-the-edge-how-to-deploy-and-optimize-models-on-nvidia-jetson)
- Source URL: https://developer.nvidia.com/blog/frontier-reasoning-reaches-the-edge-how-to-deploy-and-optimize-models-on-nvidia-jetson
- Ingested by Yomu: 2026-09-04T20:00:50.645Z

[Read original post](https://developer.nvidia.com/blog/frontier-reasoning-reaches-the-edge-how-to-deploy-and-optimize-models-on-nvidia-jetson)