Accelerating BEV Pooling on NVIDIA GPUs for Physical AI Applications

Summary

Bird's-eye-view (BEV) pooling frequently becomes an inference latency bottleneck in spatial AI pipelines due to irregular memory access patterns and scatter-reduce operations. BEVPoolV3 addresses this by tailoring kernel execution to the GPU memory regime, specifically distinguishing between DRAM-bound and L2-cache-resident working sets. The implementation reduces redundant scatter traffic via depth-outer traversal, precomputes indices to eliminate runtime integer division, minimizes duplicate depth loads, and adopts interval-owned writes to bypass atomic operations. On a canonical 49 MB nuScenes working set, BEVPoolV3 accelerates TensorRT plugin execution from 274.0 µs down to 16.4 µs on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Furthermore, profiling with Nsight Compute reveals that L2-resident scatter-reduce operators achieve optimal efficiency with FP8, avoiding the ALU instruction overhead of NVFP4 decoding.

Context

Bird's-eye-view (BEV) perception models in autonomous vehicles and robotics rely on BEV pooling to project multicamera image features into a unified top-down grid. However, BEV pooling often creates a latency bottleneck during deployment due to irregular memory access, repeated index reads, scatter-reduce operations, and hardware cache constraints across different GPU architectures.

Approach / What changed

The BEVPoolV3 workflow optimizes scatter-reduce operations by classifying whether the working set is DRAM-bound or L2-resident, then eliminating redundant scatter traffic. It implements a five-array INT32 scatter map, precomputes indices to remove runtime integer divisions, avoids atomics using interval-owned output writes, reduces duplicate depth loads per interval, and applies FP16 or FP8 specialization validated with NVIDIA Nsight Compute.

Takeaways

• On the canonical nuScenes benchmark, BEVPoolV3 reduces TensorRT plugin latency on the RTX PRO 6000 Blackwell Max-Q from 274.0 µs in V2 FP16 to 17.3 µs in FP16 and 16.4 µs in FP8.
• Working set sizing relative to L2 cache determines the kernel strategy: DRAM-bound GPUs like the RTX A6000 require byte reduction and cache-streaming stores, whereas L2-resident GPUs benefit from instruction efficiency, precomputed indices, and FP8 specialization.
• For L2-resident scatter-reduce kernels, NVFP4 performs slower than FP8 because per-element nibble extraction, value decoding, and scale folding introduce inner-loop ALU overhead that outweighs bandwidth savings.

Related reading