Loading…
Designing GPU-Accelerated Query Engines with NVIDIA GQE
NVIDIA Developer BlogMichelle Horton
Summary
GPU-accelerated query engines encounter performance constraints stemming from memory and I/O bandwidth limitations. NVIDIA introduces GQE, a reference architecture using NVIDIA cuDF, nvCOMP, and other CUDA-X libraries to accelerate SQL query execution across modern hardware like the NVIDIA GB200 NVL4. The architecture coordinates three distinct layers: a query layer consuming Substrait plans, a data layer managing in-memory table transfers, and an execution layer executing relational operator task graphs. GQE employs pipeline parallelism, partition pruning, hybrid compression, and batched asynchronous transfers to overlap data movement with GPU compute. On a non-standard 1 TB TPC-H benchmark, GQE on a single B200 GPU achieved a 7.5x total execution time speedup over dual-socket CPU configurations running DuckDB.
Context
GPU-accelerated query engines are frequently constrained by memory capacity and I/O bandwidth bottlenecks during data movement between host CPUs and GPUs.
Approach / What changed
NVIDIA developed GQE, a reference architecture comprising query, data, and execution layers built on cuDF and nvCOMP. It optimizes host-to-device transfers using pipeline parallelism across scheduling, H2D transfer, decompression, and compute stages. It also applies partition pruning, batched transfers via cudaMemcpyBatchAsync, and a hybrid compression model utilizing the Blackwell Decompression Engine alongside lightweight schemes like Cascaded encoding.
Takeaways
- GQE uses a four-stage pipeline to overlap host scheduling, host-to-device transfers, data decompression, and CUDA kernel execution across table row groups.
- The NVIDIA Blackwell Decompression Engine in the B200 GPU decompresses LZ77-based formats at up to 400 GB/s without consuming streaming multiprocessor resources.
- In a 1 TB TPC-H evaluation, GQE running on a single GB200 B200 GPU completed all 22 queries in 9.0 seconds, achieving a 7.5x aggregate speedup over dual-socket DuckDB baseline.
Related reading
ModelExpress: Distributing Model Artifacts at the Speed of Light
Distributing large language model weights and JIT kernel caches across clusters imposes high latency on cold starts, autoscaling, and reinforcement learning loops. NVIDIA ModelExpress minimizes this delay by identifying existing compatible weight replicas across the cluster rather than treating every worker as an independent cold start. Initial replicas bootstrap by streaming directly from remote object storage into GPU memory or bypassing host staging using GPUDirect Storage. Subsequent replicas load pre-processed weights and kernel caches directly from serving peers over peer-to-peer RDMA via the NVIDIA Inference Xfer Library. In testing on DeepSeek-V4 Pro, this peer-to-peer artifact distribution transferred weights and caches in under ten seconds, reducing overall replica startup time from eight minutes to one minute and forty-four seconds.
Elizabeth Goodman