---
title: "NPU"
description: "2 posts about NPU, summarised, each linking to the original."
---

# NPU
> 2 posts about NPU, summarised, each linking to the original.

## Articles

### [MediaTek NPU and LiteRT: Powering the next generation of on-device AI](https://yomu.fyi/post/mediatek-npu-and-litert-powering-the-next-generation-of-on-device-ai.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Lu Wang, Arian Arfaian, Luke Boyer
- Published: Dec 8, 2025
- Type: Announcement

MediaTek’s diverse NPU hardware and specialized compilation requirements have made on-device AI deployment difficult, especially for sophisticated generative models. The LiteRT NeuroPilot Accelerator, described as a ground-up successor to the TFLite NeuroPilot delegate, provides a unified API over MediaTek NPUs and supports both offline Ahead-of-Time (AOT) and on-device compilation. Its workflow can compile models for target SoCs, package Android assets and runtimes for Google Play’s Play for On-device AI, and invoke NPU inference through LiteRT with GPU or CPU fallback options. A new C++ API and Native Hardware Buffer Interoperability enable zero-copy transfers from AHardwareBuffer and OpenGL/OpenCL buffers, supporting real-time camera and video pipelines. On supported MediaTek chipsets, listed Gemma and Qwen models receive NPU-specific optimizations; the post reports up to 12x CPU and 10x GPU acceleration, including over 1600 tokens/sec prefill and 28 tokens/sec decode for Gemma 3n E2B.


### [Unlocking Peak Performance on Qualcomm NPU with LiteRT](https://yomu.fyi/post/unlocking-peak-performance-on-qualcomm-npu-with-litert.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Lu Wang, Weiyi Wang, Andrew Zhang
- Published: Nov 24, 2025
- Type: Announcement

Google introduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, developed with Qualcomm to replace the previous TFLite QNN delegate and simplify NPU deployment across supported Qualcomm SoCs. It integrates vendor compilers and runtimes behind a unified workflow, supports ahead-of-time and on-device compilation, and can fall back to GPU or CPU when needed. Across 72 canonical vision, audio, and NLP models, the accelerator supports 90 LiteRT ops, enables full NPU delegation for 64, and reports up to 100x CPU and 10x GPU speedups. On Snapdragon 8 Elite Gen 5, FastVLM-0.5B uses int8 weights, int16 activations, and specialized transformer kernels to reach 0.12 second time-to-first-token, over 11,000 prefill tokens per second, and over 100 decode tokens per second.