Topic

cuDF

1 post about cuDF. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

NVIDIA
How to Rewrite Apache Spark UDFs for GPU Acceleration with AI Agents
User-defined functions (UDFs) in Apache Spark break the declarative abstraction, forcing GPU-accelerated pipelines to fall back to CPU execution and incur data transfer overhead. NVIDIA cuDF addresses this with GPU parallelism, but UDFs remain a bottleneck due to their opaque code. The post describes three approaches to rewrite UDFs into GPU-accelerable forms: SQL expressions, the RapidsUDF interface using cuDF columnar operations, or custom CUDA kernels via JNI. The cuDF skills for Apache Spark automate this process with four agent-guided feedback loops for testing, conversion, benchmarking, and optimization, providing a project template and best practices. A production deployment converted over 30 UDFs, achieving a 22x speedup on a 600-billion-row workload, reducing operator time from 274.55 to 12.78 hours.

The best engineering reads, once a week.

Topic

1 post about cuDF. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

NVIDIA
How to Rewrite Apache Spark UDFs for GPU Acceleration with AI Agents
User-defined functions (UDFs) in Apache Spark break the declarative abstraction, forcing GPU-accelerated pipelines to fall back to CPU execution and incur data transfer overhead. NVIDIA cuDF addresses this with GPU parallelism, but UDFs remain a bottleneck due to their opaque code. The post describes three approaches to rewrite UDFs into GPU-accelerable forms: SQL expressions, the RapidsUDF interface using cuDF columnar operations, or custom CUDA kernels via JNI. The cuDF skills for Apache Spark automate this process with four agent-guided feedback loops for testing, conversion, benchmarking, and optimization, providing a project template and best practices. A production deployment converted over 30 UDFs, achieving a 22x speedup on a 600-billion-row workload, reducing operator time from 274.55 to 12.78 hours.

The best engineering reads, once a week.