Accelerating BEV Pooling on NVIDIA GPUs for Physical AI Applications

Summary

Bird's-eye-view (BEV) pooling frequently becomes an inference latency bottleneck in spatial AI pipelines due to irregular memory access patterns and scatter-reduce operations. BEVPoolV3 addresses this by tailoring kernel execution to the GPU memory regime, specifically distinguishing between DRAM-bound and L2-cache-resident working sets. The implementation reduces redundant scatter traffic via depth-outer traversal, precomputes indices to eliminate runtime integer division, minimizes duplicate depth loads, and adopts interval-owned writes to bypass atomic operations. On a canonical 49 MB nuScenes working set, BEVPoolV3 accelerates TensorRT plugin execution from 274.0 µs down to 16.4 µs on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Furthermore, profiling with Nsight Compute reveals that L2-resident scatter-reduce operators achieve optimal efficiency with FP8, avoiding the ALU instruction overhead of NVFP4 decoding.

Takeaways

1. 1

   On the canonical nuScenes benchmark, BEVPoolV3 reduces TensorRT plugin latency on the RTX PRO 6000 Blackwell Max-Q from 274.0 µs in V2 FP16 to 17.3 µs in FP16 and 16.4 µs in FP8.

2. 2

   Working set sizing relative to L2 cache determines the kernel strategy: DRAM-bound GPUs like the RTX A6000 require byte reduction and cache-streaming stores, whereas L2-resident GPUs benefit from instruction efficiency, precomputed indices, and FP8 specialization.

3. 3

   For L2-resident scatter-reduce kernels, NVFP4 performs slower than FP8 because per-element nibble extraction, value decoding, and scale folding introduce inner-loop ALU overhead that outweighs bandwidth savings.