Control How Your GPU Shares Work with Green Contexts

Summary

Green contexts, available in the CUDA Driver API since 12.4 and the Runtime API since 13.1, let applications explicitly partition GPU execution resources such as SMs and workqueues. They are lightweight to create and destroy, do not implicitly synchronize unrelated work, and allow provisioning workqueues to reduce false dependencies. Streams are created from a green context rather than relying on implicit device state, making targeting explicit. Benchmarks on a Blackwell GPU show a critical kernel completing in 0.007 ms with green context partitioning, compared to 0.140 ms with high-priority streams and 3.727 ms with equal priority. The tradeoff is fewer SMs for the bulk workload, but green contexts are opt-in and can be adopted incrementally alongside existing CUDA code.

Takeaways

1. 1

   Green contexts allow explicit SM partitioning, enabling latency-sensitive kernels to run on dedicated SMs without waiting for bulk kernel blocks to drain, achieving sub-millisecond latency (0.007 ms vs 0.140 ms with stream priority alone on Blackwell GPU).

2. 2

   Green contexts also provision workqueue resources to reduce false dependencies between independent stream-ordered workloads, avoiding unintended serialization even when sufficient execution resources are available.

3. 3

   Green contexts are opt-in and additive; existing CUDA applications can continue to target the full device unchanged and adopt green contexts incrementally where finer control over execution is needed.

FAQ