Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

Summary

Hugging Face’s WebAI team is releasing @huggingface/kernels, a JavaScript loader for optimized WebGPU kernels hosted on the Hub, alongside an initial collection of 207 Apache-2.0 kernels. Each kernel repository packages a versioned contract, metadata, correctness cases, benchmark cases, and parameterized WGSL shader templates, while the loader derives outputs and selects variants for a request and device. Fleet adds browser-based correctness and performance testing, with consent-based private evidence used to identify failures, compare variants, and guide optimization. In comparisons against ORT WebGPU on an Apple M4, matching cases showed a 2.57x geometric-mean speedup and a 1.90x median speedup, though the measurements cover individual GPU operations and exclude setup costs. The project is also intended as a shared foundation for browser inference and future integration with higher-level tooling.

Takeaways

1. 1

   Each kernel’s manifest defines its inputs, outputs, attributes, type constraints, and shape derivation rules; the loader uses that contract to allocate outputs and expose a stable JavaScript-facing interface.

2. 2

   Against ORT WebGPU on an Apple M4, 809 matching cases produced 629 wins, 176 losses, and 4 ties, with a 2.57x geometric-mean speedup and 1.90x median speedup.

3. 3

   Fleet can collect private correctness and performance evidence, with consent, to reveal device-specific failures, compare kernel variants, and improve optimization and selection rules.