---
title: "MediaTek NPU and LiteRT: Powering the next generation of on-device AI"
description: "MediaTek’s diverse NPU hardware and specialized compilation requirements have made on-device AI deployment difficult, especially for sophisticated generative models. The LiteRT NeuroPilot Accelerator, described as a ground-up successor to the TFLite NeuroPilot delegate, provides a unified API over MediaTek NPUs and supports both offline Ahead-of-Time (AOT) and on-device compilation. Its workflow can compile models for target SoCs, package Android assets and runtimes for Google Play’s Play for On-device AI, and invoke NPU inference through LiteRT with GPU or CPU fallback options. A new C++ API and Native Hardware Buffer Interoperability enable zero-copy transfers from AHardwareBuffer and OpenGL/OpenCL buffers, supporting real-time camera and video pipelines. On supported MediaTek chipsets, listed Gemma and Qwen models receive NPU-specific optimizations; the post reports up to 12x CPU and 10x GPU acceleration, including over 1600 tokens/sec prefill and 28 tokens/sec decode for Gemma 3n E2B."
---

# MediaTek NPU and LiteRT: Powering the next generation of on-device AI

[Google](https://yomu.fyi/company/google) · Lu Wang, Arian Arfaian, Luke Boyer · Dec 8, 2025

**Type:** Announcement

## Summary

MediaTek’s diverse NPU hardware and specialized compilation requirements have made on-device AI deployment difficult, especially for sophisticated generative models. The LiteRT NeuroPilot Accelerator, described as a ground-up successor to the TFLite NeuroPilot delegate, provides a unified API over MediaTek NPUs and supports both offline Ahead-of-Time (AOT) and on-device compilation. Its workflow can compile models for target SoCs, package Android assets and runtimes for Google Play’s Play for On-device AI, and invoke NPU inference through LiteRT with GPU or CPU fallback options. A new C++ API and Native Hardware Buffer Interoperability enable zero-copy transfers from AHardwareBuffer and OpenGL/OpenCL buffers, supporting real-time camera and video pipelines. On supported MediaTek chipsets, listed Gemma and Qwen models receive NPU-specific optimizations; the post reports up to 12x CPU and 10x GPU acceleration, including over 1600 tokens/sec prefill and 28 tokens/sec decode for Gemma 3n E2B.

## Context

The NPU ecosystem includes hundreds of SoC variants, while existing on-device ML infrastructure is primarily designed for CPUs and GPUs. Developers have had to manage different compilers, SDKs, runtimes, compiled models, and specialized optimizations, making NPU deployment complex and ad hoc.

## Approach / What changed

The LiteRT NeuroPilot Accelerator directly integrates with the NeuroPilot compiler and runtime. It provides a unified API, optional AOT or on-device compilation, Android AI Pack distribution through Play for On-device AI, NPU inference with GPU or CPU fallback, a simplified C++ API, and Native Hardware Buffer Interoperability for zero-copy data exchange.

## Takeaways

- AOT compilation is recommended for larger models because it reduces on-device initialization time; on-device compilation supports platform-agnostic distribution but adds first-run cost, taking over a minute for a model such as Gemma 3 270M.
- LiteRT can package model assets and required runtime libraries into an Android AI Pack, which Play for On-device AI analyzes and delivers to compatible devices.
- Native Hardware Buffer Interoperability supports zero-copy transfers from AHardwareBuffer and OpenGL/OpenCL buffers, avoiding CPU conversions in camera and video pipelines.

**Tags:** [Android](https://yomu.fyi/topic/android), [Generative AI](https://yomu.fyi/topic/generative-ai), [LiteRT](https://yomu.fyi/topic/litert), [NPU](https://yomu.fyi/topic/npu)

- Source: [Google](https://developers.googleblog.com/mediatek-npu-and-litert-powering-the-next-generation-of-on-device-ai)
- Source URL: https://developers.googleblog.com/mediatek-npu-and-litert-powering-the-next-generation-of-on-device-ai
- Ingested by Yomu: 2026-09-09T14:27:32.665Z

[Read original post](https://developers.googleblog.com/mediatek-npu-and-litert-powering-the-next-generation-of-on-device-ai)