LiteRT.js, Google's high performance Web AI Inference

Summary

Google announces LiteRT.js, a JavaScript binding of LiteRT that runs .tflite AI and ML models directly in web browsers. It targets local inference for JavaScript and TypeScript applications, with privacy benefits, zero server costs, and low latency, while providing smoother deployment for existing .tflite models. Through WebAssembly, it exposes LiteRT’s optimized runtime across XNNPACK CPU, ML Drift with WebGPU, and experimental WebNN for NPUs; LiteRT Torch supports one-step PyTorch conversion, and AI Edge Quantizer enables layer-specific quantization. The initial release includes the @litertjs/core npm package, demos, and examples covering YOLO26, depth estimation, and Real-ESRGAN upscaling. In controlled benchmarks on a 2024 Apple MacBook Pro with M4, it outperformed other web runtimes by up to 3x, while WebGPU or WebNN delivered 5–60x speedups over CPU execution; results may vary with hardware, thermal conditions, and browser drivers.

Takeaways

1. 1

   LiteRT.js uses XNNPACK for CPU inference, ML Drift with WebGPU for GPU inference, and experimental WebNN to target NPUs.

2. 2

   LiteRT Torch converts PyTorch models in a single step, while AI Edge Quantizer supports configurable quantization schemes across model layers.

3. 3

   Controlled tests on a 2024 M4 Apple MacBook Pro reported up to 3x speedups over other web runtimes and 5–60x speedups from WebGPU or WebNN over CPU execution.