Topic

FP8

1 post about FP8. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Hugging Face
MPS in PyTorch 2.15
PyTorch 2.15 brings FP8 support to MPS, storing float8_e4m3fn values in half the space of BF16 and enabling scaled matrix multiplication directly on the GPU with tensorwise or rowwise scaling. Language-model decoding accelerates significantly: mixture-of-experts models decode 5–6.5× faster thanks to a native grouped matmul with a dedicated decoding path—on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B reaches 61.3 tokens/s, up from 9.4. Dense models benefit from specialized matrix-vector kernels for single-row linear layers, and softmax over the vocabulary is about 8× faster. Video generation memory usage drops dramatically: Wan2.2’s VAE decode now peaks at 8 GiB instead of 270 GiB. The Hugging Face team runs real models to find and fix MPS shortcomings. The release also adds complex LU and triangular solves, faster small-matrix inverses and QR decomposition, public MPS streams, and requires macOS 15 or later, with many operations moved from MPSGraph to dedicated Metal kernels.

The best engineering reads, once a week.

Topic

1 post about FP8. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Hugging Face
MPS in PyTorch 2.15
PyTorch 2.15 brings FP8 support to MPS, storing float8_e4m3fn values in half the space of BF16 and enabling scaled matrix multiplication directly on the GPU with tensorwise or rowwise scaling. Language-model decoding accelerates significantly: mixture-of-experts models decode 5–6.5× faster thanks to a native grouped matmul with a dedicated decoding path—on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B reaches 61.3 tokens/s, up from 9.4. Dense models benefit from specialized matrix-vector kernels for single-row linear layers, and softmax over the vocabulary is about 8× faster. Video generation memory usage drops dramatically: Wan2.2’s VAE decode now peaks at 8 GiB instead of 270 GiB. The Hugging Face team runs real models to find and fix MPS shortcomings. The release also adds complex LU and triangular solves, faster small-matrix inverses and QR decomposition, public MPS streams, and requires macOS 15 or later, with many operations moved from MPSGraph to dedicated Metal kernels.

The best engineering reads, once a week.