---
title: "CUDA IPC"
description: "1 posts about CUDA IPC, summarised, each linking to the original."
---

# CUDA IPC
> 1 posts about CUDA IPC, summarised, each linking to the original.

## Articles

### [Boosting multimodal inference performance by >10% with a single Python dictionary](https://yomu.fyi/post/boosting-multimodal-inference-performance-by-10-with-a-single-python-d.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: May 4, 2026
- Type: Problem & solution

Multimodal vision-language models pose new challenges for inference engines like SGLang. Profiling with py-spy revealed that the scheduler’s process\_input\_requests consumed about 13% of CPU time, with a quarter of that spent on repeated calls to torch.UntypedStorage.\_new\_shared\_cuda for CUDA IPC handle book-keeping. The solution was a simple Python dictionary cache for pool handles, eliminating redundant opens. On a Qwen2.5-VL-3B-Instruct benchmark with a single H100, throughput increased 16.2% from 22.2 to 25.7 requests per second, mean time-to-first-token dropped 13.2%, and mean time-per-output-token dropped 17.2%. The improvement is merged in SGLang v0.5.10 and applies to any multimodal model using SGLang’s CUDA IPC transport.