Hardware-Rooted AI Security That Won’t Slow You Down

Summary

NVIDIA Confidential Computing secures enterprise data, proprietary model weights, and the model itself during active AI inference. The architecture integrates silicon-level private signing keys fused during manufacturing, encrypted multi-GPU NVLink, and remote attestation through the NVIDIA Remote Attestation Service alongside CPU Trusted Execution Environments. To mitigate overhead from secure work submission latency and encrypted host-to-device transfers, software optimizations incorporate CC-safe autotuner timing in FlashInfer, asynchronous copy workers in SGLang, and piecewise CUDA graph replay. Benchmarks on an HGX B300 system running the Qwen 3.5 397B-A17B model at FP8 precision show inference performance reaches up to 98% of non-confidential baselines across tested concurrency levels.

Takeaways

1. 1

   Blackwell GPUs utilize a factory-fused private signing key and remote attestation via NRAS to verify Trusted Execution Environments before deploying model decryption keys.

2. 2

   Performance overhead in Confidential Computing stems primarily from secure work submission latency and reduced encrypted host-to-device CPU-to-GPU bandwidth.

3. 3

   Benchmarking Qwen 3.5 397B-A17B on an HGX B300 with Intel TDX demonstrated throughput and median TPOT overhead within single-digit percentage deltas compared to baseline non-CC execution across multiple concurrency levels.