---
title: "Neural Audio Encoders"
description: "1 posts about Neural Audio Encoders, summarised, each linking to the original."
---

# Neural Audio Encoders
> 1 posts about Neural Audio Encoders, summarised, each linking to the original.

## Articles

### [Compressing Streaming Neural Audio Encoders via Latent-Space Distillation](https://yomu.fyi/post/compressing-streaming-neural-audio-encoders-via-latent-space-distillat.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Sep 24, 2026
- Type: Problem & solution

Apple’s on-device dictation tokenizer maps waveform to latent representations for a sparsely activated language model, and compressing it reduces memory and latency demands. The authors compress it via latent-space distillation, using the teacher’s pre-quantizer latent as the supervision target rather than tokens or output distributions. A student encoder is trained to regress the teacher’s per-frame latent with a squared-error objective, and an affine layer absorbs width mismatch. This recipe works for both token interfaces and for tokenizers pretrained alone or jointly with a language model, requiring no fine-tuning. At 2.8x compression, the student stays within 1.9% relative WER of its teacher on five of six pairs and outperforms an independently trained tokenizer of identical capacity by 3.9% relative.