---
title: "Unlocking Peak Performance on Qualcomm NPU with LiteRT"
description: "Google introduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, developed with Qualcomm to replace the previous TFLite QNN delegate and simplify NPU deployment across supported Qualcomm SoCs. It integrates vendor compilers and runtimes behind a unified workflow, supports ahead-of-time and on-device compilation, and can fall back to GPU or CPU when needed. Across 72 canonical vision, audio, and NLP models, the accelerator supports 90 LiteRT ops, enables full NPU delegation for 64, and reports up to 100x CPU and 10x GPU speedups. On Snapdragon 8 Elite Gen 5, FastVLM-0.5B uses int8 weights, int16 activations, and specialized transformer kernels to reach 0.12 second time-to-first-token, over 11,000 prefill tokens per second, and over 100 decode tokens per second."
---

# Unlocking Peak Performance on Qualcomm NPU with LiteRT

[Google](https://yomu.fyi/company/google) · Lu Wang, Weiyi Wang, Andrew Zhang · Nov 24, 2025

**Type:** Announcement

## Summary

Google introduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, developed with Qualcomm to replace the previous TFLite QNN delegate and simplify NPU deployment across supported Qualcomm SoCs. It integrates vendor compilers and runtimes behind a unified workflow, supports ahead-of-time and on-device compilation, and can fall back to GPU or CPU when needed. Across 72 canonical vision, audio, and NLP models, the accelerator supports 90 LiteRT ops, enables full NPU delegation for 64, and reports up to 100x CPU and 10x GPU speedups. On Snapdragon 8 Elite Gen 5, FastVLM-0.5B uses int8 weights, int16 activations, and specialized transformer kernels to reach 0.12 second time-to-first-token, over 11,000 prefill tokens per second, and over 100 decode tokens per second.

## Context

The post addresses GPU performance bottlenecks in interactive on-device GenAI workloads, particularly when a compute-intensive model runs alongside live camera processing. It presents NPU acceleration as a way to run heavy AI processing in parallel with GPU rendering and CPU main-thread logic while improving power efficiency.

## Approach / What changed

The LiteRT Qualcomm AI Engine Direct Accelerator unifies Qualcomm NPU deployment through LiteRT, supports ahead-of-time and on-device compilation across SoCs, and provides CPU or GPU fallback. The workflow includes optional AOT compilation, Google Play for On-device AI delivery, runtime library setup, and LiteRT Runtime API inference. Specialized kernels and quantization target LLM and GenAI performance.

## Takeaways

- The accelerator supports 90 LiteRT ops and fully delegates 64 of 72 benchmarked vision, audio, and NLP models to the NPU.
- Across the benchmark, NPU acceleration reaches up to 100x the CPU performance and 10x the GPU performance; on Snapdragon 8 Elite Gen 5, more than 56 models run under 5ms on NPU.
- FastVLM-0.5B uses int8 weight and int16 activation quantization plus specialized attention kernels, achieving 0.12-second TTFT, over 11,000 prefill tokens per second, and over 100 decode tokens per second.

**Tags:** [LiteRT](https://yomu.fyi/topic/litert), [NPU](https://yomu.fyi/topic/npu), [Performance](https://yomu.fyi/topic/performance), [Quantization](https://yomu.fyi/topic/quantization)

- Source: [Google](https://developers.googleblog.com/unlocking-peak-performance-on-qualcomm-npu-with-litert)
- Source URL: https://developers.googleblog.com/unlocking-peak-performance-on-qualcomm-npu-with-litert
- Ingested by Yomu: 2026-09-09T14:27:28.978Z

[Read original post](https://developers.googleblog.com/unlocking-peak-performance-on-qualcomm-npu-with-litert)