Custom Kernels for All from Codex and Claude

Summary

Writing CUDA kernels that integrate with diffusers and transformers involves GPU-specific optimization, library integration, PyTorch registration, and environment compatibility. It presents a cuda-kernels agent skill that packages architecture guides, integration patterns, kernel templates, troubleshooting material, benchmarks, and Kernel Hub examples for coding agents. Claude and Codex used it to generate RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for LTX-Video in diffusers, and an RMSNorm kernel for Qwen3-8B in transformers, including CUDA source, PyTorch bindings, build configuration, and benchmark scripts. On H100 80GB HBM3, the LTX-Video RMSNorm tests averaged 1.88x speedup and the optimized pipeline with torch.compile reached 1.43x the listed baseline, while Qwen3-8B RMSNorm averaged 1.94x and reached 2.47x at 8192 tokens. The skill supports packaging kernels for precompiled distribution through the Hub, where get_kernel selects a matching binary across Python, PyTorch, and CUDA variants.

Takeaways

1. 1

   The generated LTX-Video RMSNorm kernel averaged 1.88x speedup over the PyTorch baseline across the listed H100 BF16 shapes, with 34.7% of H100 theoretical bandwidth.

2. 2

   For Qwen3-8B, RMSNorm speedup increased from 1.58x at 128 tokens to 2.47x at 8192 tokens; the post reports 1.94x average speedup and 22.3% theoretical bandwidth efficiency.

3. 3

   In the LTX-Video end-to-end table, optimized kernels with torch.compile reached 18.45 iterations per second and a listed 1.43x speedup, while RMSNorm represented about 5% of total compute.