---
title: "When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving"
description: "Encode-prefill-decode (EPD) disaggregation separates a multimodal model’s vision encoder from LLM prefill and decode, addressing contention when image or video processing delays language-model work. Using NVIDIA Dynamo, the post compares aggregated serving with colocated encoder workers and a heterogeneous, disaggregated encoder tier, with vision embeddings transferred to PD workers through NIXL over UCX RC/TCP Ethernet. EPD is most useful for media-heavy requests, short-to-medium outputs, smaller or quantized models, and mixed text/multimodal traffic; long decode sequences or light media processing reduce its value. In the reported image-heavy test, colocated and heterogeneous EPD reduced TTFT by 58% and 50%, while the heterogeneous tier served 70% more traffic at the same under-100-ms ITL SLO. Mixed-traffic measurements also reduced mean TTFT for text requests from 92.3 to 53.3 ms."
---

# When to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Tanya Lenz · Sep 9, 2026

**Type:** Explainer

## Summary

Encode-prefill-decode (EPD) disaggregation separates a multimodal model’s vision encoder from LLM prefill and decode, addressing contention when image or video processing delays language-model work. Using NVIDIA Dynamo, the post compares aggregated serving with colocated encoder workers and a heterogeneous, disaggregated encoder tier, with vision embeddings transferred to PD workers through NIXL over UCX RC/TCP Ethernet. EPD is most useful for media-heavy requests, short-to-medium outputs, smaller or quantized models, and mixed text/multimodal traffic; long decode sequences or light media processing reduce its value. In the reported image-heavy test, colocated and heterogeneous EPD reduced TTFT by 58% and 50%, while the heterogeneous tier served 70% more traffic at the same under-100-ms ITL SLO. Mixed-traffic measurements also reduced mean TTFT for text requests from 92.3 to 53.3 ms.

## Context

Multimodal requests must preprocess media and run the vision transformer before LLM prefill. In aggregated serving, these stages share a worker and scheduling domain, so media-heavy requests can delay their own prefill and contend with concurrent prefill and decode work; text-only requests in mixed batches can also wait behind vision processing.

## Approach / What changed

NVIDIA Dynamo separates encoder and prefill/decode worker roles so the stages can batch, schedule, and scale independently. Encoder workers produce vision embeddings, which PD workers consume through NIXL. The post compares aggregated serving with colocated encoders on the same GPUs and disaggregated encoders on a separate, lower-cost GPU tier.

## Takeaways

- On homogeneous clusters, colocated encoder workers avoid dedicating an entire GPU to the relatively lightweight vision encoder; heterogeneous tiers are more useful when lower-tier GPUs can host encoder work.
- EPD does not optimize the 43% of TTFT spent before the ViT starts; the post points to parallel media decoding, embedding cache, and multimodal KV routing for other pipeline stages.
- With parallel media decoding enabled, an encoder-only benchmark reported mean latency falling from 281.3 to 207.0 ms and P99 latency from 752.1 to 581.7 ms at matched throughput.

**Tags:** [GPU](https://yomu.fyi/topic/gpu), [Inference](https://yomu.fyi/topic/inference), [Model Serving](https://yomu.fyi/topic/model-serving), [Multimodal](https://yomu.fyi/topic/multimodal), [Performance](https://yomu.fyi/topic/performance)

- Source: [NVIDIA](https://developer.nvidia.com/blog/when-to-use-encode-prefill-decode-disaggregation-to-accelerate-multimodal-model-serving)
- Source URL: https://developer.nvidia.com/blog/when-to-use-encode-prefill-decode-disaggregation-to-accelerate-multimodal-model-serving
- Ingested by Yomu: 2026-09-10T00:05:23.153Z

[Read original post](https://developer.nvidia.com/blog/when-to-use-encode-prefill-decode-disaggregation-to-accelerate-multimodal-model-serving)