Loading…
Hardware-Rooted AI Security That Won’t Slow You Down
NVIDIA Developer BlogElizabeth Goodman
Summary
NVIDIA Confidential Computing secures enterprise data, proprietary model weights, and the model itself during active AI inference. The architecture integrates silicon-level private signing keys fused during manufacturing, encrypted multi-GPU NVLink, and remote attestation through the NVIDIA Remote Attestation Service alongside CPU Trusted Execution Environments. To mitigate overhead from secure work submission latency and encrypted host-to-device transfers, software optimizations incorporate CC-safe autotuner timing in FlashInfer, asynchronous copy workers in SGLang, and piecewise CUDA graph replay. Benchmarks on an HGX B300 system running the Qwen 3.5 397B-A17B model at FP8 precision show inference performance reaches up to 98% of non-confidential baselines across tested concurrency levels.
Context
Concerns surrounding data privacy, sovereignty, and securing data while in use during inference impede enterprise AI adoption.
Approach / What changed
NVIDIA implements Confidential Computing across Blackwell GPUs using fused hardware keys, remote attestation via NRAS with CPU TEEs, NVLink encryption, and software optimizations across FlashInfer and SGLang.
Takeaways
- Blackwell GPUs utilize a factory-fused private signing key and remote attestation via NRAS to verify Trusted Execution Environments before deploying model decryption keys.
- Performance overhead in Confidential Computing stems primarily from secure work submission latency and reduced encrypted host-to-device CPU-to-GPU bandwidth.
- Benchmarking Qwen 3.5 397B-A17B on an HGX B300 with Intel TDX demonstrated throughput and median TPOT overhead within single-digit percentage deltas compared to baseline non-CC execution across multiple concurrency levels.
Related reading
Building Faster Cryptography with Carryless Multiplication in NVIDIA CUDA 13.3
NVIDIA CUDA 13.3 introduces clmad, a native PTX carryless multiply-accumulate instruction for Ampere and newer GPU architectures. Prior to this release, GPUs lacked dedicated hardware for carryless multiplication, requiring developers to emulate binary extension field arithmetic with ALU operations or bitsliced circuits. The new primitive performs 64-bit by 64-bit carryless multiplication into 128-bit results with accumulation, allowing full 128-bit field multiplications via Karatsuba in six instructions. Benchmarks demonstrate that clmad achieves up to 6.3 TB/s throughput for GHASH on the NVIDIA B200, representing an 18.8x improvement over bitslicing. Additionally, sum-check protocol latency in zero-knowledge proving workloads decreases by up to 13x on the B200 and 4x on the RTX 5090.
Michelle Horton