---
title: "Native Model Authoring for Edge with LiteRT Tensor API"
description: "The LiteRT Tensor API is a new lightweight authoring layer that allows developers to handcraft models in C++ with full control over graph structure and operations. The same graph runs through LiteRT on CPU, GPU, or NPU, or directly on CPU via XNNPACK, without rewriting for each hardware target. The API enables direct weight ingestion from model checkpoints like Hugging Face .safetensors files and provides tensor-centric buffer interop to keep intermediate data on accelerators with zero copies. A reference implementation of Meta’s Segment Anything Model 2 (SAM 2) authored entirely with the API achieved over 30 frames per second at 512x512 on WebGPU in the browser. The API also provides structured primitives for AI coding agents to generate, rewire, and tune model graphs automatically."
---

# Native Model Authoring for Edge with LiteRT Tensor API

[Google](https://yomu.fyi/company/google) · Ping Yu, Quentin Khan, Chintan Parikh, Lu Wang · Oct 19, 2026

**Type:** Announcement

## Summary

The LiteRT Tensor API is a new lightweight authoring layer that allows developers to handcraft models in C++ with full control over graph structure and operations. The same graph runs through LiteRT on CPU, GPU, or NPU, or directly on CPU via XNNPACK, without rewriting for each hardware target. The API enables direct weight ingestion from model checkpoints like Hugging Face .safetensors files and provides tensor-centric buffer interop to keep intermediate data on accelerators with zero copies. A reference implementation of Meta’s Segment Anything Model 2 (SAM 2) authored entirely with the API achieved over 30 frames per second at 512x512 on WebGPU in the browser. The API also provides structured primitives for AI coding agents to generate, rewire, and tune model graphs automatically.

## Context

Deploying on-device AI requires meeting tight latency and memory budgets. The standard conversion path (e.g., PyTorch to LiteRT) is recommended for most projects, but developers sometimes need full control over graph representation and optimization for new architectures or custom lowering logic. Handcrafting models in native C++ or shader languages offers control but is tightly coupled to specific backends and requires significant engineering for cross-platform portability and buffer management.

## Approach / What changed

The LiteRT Tensor API is a lightweight authoring layer that allows building computation graphs op by op in C++, with cross-platform portability via LiteRT (CPU, GPU, NPU) or direct CPU execution via XNNPACK. It provides direct weight ingestion, tensor-centric buffer interop, and building blocks for AI coding agents.

## Takeaways

- The LiteRT Tensor API enables handcrafting models in C++ with full control over graph structure and operations, while maintaining cross-platform portability across CPU, GPU, and NPU.
- The API supports direct weight ingestion from model checkpoints (e.g., .safetensors) without conversion scripts, and allows tensor-centric buffer interop to keep data on accelerators with zero copies.
- A coding agent (Antigravity) used the Tensor API to optimize the SAM 2 model, more than tripling its frame rate by rewriting per-frame decisions as GPU-friendly operations, each change checked against a reference.

## FAQ

### How does the LiteRT Tensor API differ from the standard conversion path?

The standard conversion path (e.g., PyTorch to LiteRT) is the recommended default for most projects. The Tensor API is for developers who need low-level control over graph structure and optimization, allowing handcrafting in C++ with cross-platform portability.

### What backends does the Tensor API support?

The same C++ graph can run through LiteRT on CPU, GPU (via open-source ML Drift), and NPU, or directly on CPU through XNNPACK without a model file. Tensors can be tagged for both backends.

### How can AI coding agents use the Tensor API?

The API provides structured primitives for AI coding agents to generate, rewire, and tune model graphs automatically. The Antigravity agent used it to optimize the SAM 2 model by rewriting per-frame decisions as GPU-friendly operations, checking each change against a reference.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [LiteRT](https://yomu.fyi/topic/litert), [SAM 2](https://yomu.fyi/topic/sam-2), [WebGPU](https://yomu.fyi/topic/webgpu)

- Source: [Google](https://developers.googleblog.com/native-model-authoring-for-edge-with-litert-tensor-api)
- Source URL: https://developers.googleblog.com/native-model-authoring-for-edge-with-litert-tensor-api
- Ingested by Yomu: 2026-10-19T18:00:18.321Z

[Read original post](https://developers.googleblog.com/native-model-authoring-for-edge-with-litert-tensor-api)