Transformers.js v4: Now Available on NPM!

Summary

Transformers.js v4 is now available on NPM, marking a major release after a year of development. It introduces a C++-rewritten WebGPU runtime tested across about 200 supported model architectures, enabling the same JavaScript code to run WebGPU-accelerated models in browsers, Node, Bun, Deno, and desktop applications. Specialized ONNX Runtime Contrib Operators support new export strategies for large language models, and the post reports a roughly fourfold speedup for BERT-based embedding models using MultiHeadAttention. The release also restructures the repository as a pnpm-workspaces monorepo, modularizes model definitions, moves examples to a separate repository, and switches from Webpack to esbuild, reducing build times from two seconds to 200 milliseconds and shrinking transformers.web.js by 53%. New APIs cover model asset inspection, caching, environment fetch controls, logging, and standalone tokenization, while added architectures include Mamba, MLA, and MoE models compatible with WebGPU.

Takeaways

1. 1

   The new WebGPU runtime allows Transformers.js code to run hardware-accelerated models across browsers, Node, Bun, Deno, and desktop applications.

2. 2

   Using the com.microsoft.MultiHeadAttention operator produced a reported roughly 4x speedup for BERT-based embedding models.

3. 3

   The esbuild migration reduced build times from 2 seconds to 200 milliseconds, while transformers.web.js became 53% smaller.