CCCL Runtime: A Modern C++ Runtime for CUDA

Summary

NVIDIA CCCL runtime introduces modern C++ abstractions for fundamental CUDA programming concepts starting in CUDA Toolkit 13.2 and CCCL 3.2. Designed as an alternative to the traditional CUDA runtime API, the library eliminates implicit global state by requiring explicit dependencies, such as binding streams directly to specific device references. The API enforces strong typing through owning types and non-owning reference types, omits default stream usage to ensure all streams remain non-blocking, and handles errors via standard C++ exceptions. In addition, CCCL runtime simplifies kernel launches through kernel functors that enable automatic template argument deduction and automatically transforms owning buffer arguments into device spans.

Takeaways

1. 1

   CCCL runtime replaces implicit global state with explicit dependencies, requiring streams to be constructed with a specific device reference and making all created streams non-blocking.

2. 2

   The runtime uses paired owning types and non-owning ref types, allowing seamless conversions and lifetime management alongside legacy raw handles like cudaStream_t.

3. 3

   Kernel functors with device call operators enable automatic template argument deduction during launch, while buffers passed to cuda::launch automatically transform into cuda::std::span.