---
title: "DeepSeek-V4: a million-token context that agents can actually use"
description: "DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M."
---

# DeepSeek-V4: a million-token context that agents can actually use

[Hugging Face](https://yomu.fyi/company/hugging-face) · ben burtenshaw · Apr 24, 2026

**Type:** Explainer

## Summary

DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M.

## Context

Long-running agent workflows can fail when reasoning is discarded across user turns, context growth exceeds the budget, KV cache memory fills the GPU, or tool-call round trips degrade over many steps. The post positions efficient large-context inference and agent-specific training as responses to these failure modes.

## Approach / What changed

DeepSeek V4 alternates CSA and HCA attention layers, compresses KV entries, and uses FP8, BF16, and FP4 storage choices to reduce inference and cache costs. It preserves reasoning across tool-call turns, adds an XML-based tool-call schema with dedicated tokens, and trains agent behavior in DSec, a Rust sandbox platform supporting function calls, containers, microVMs, and full VMs.

## Takeaways

- V4-Pro uses 27% of DeepSeek-V3.2's single-token inference FLOPs and 10% of its KV cache at 1M tokens; V4-Flash uses 10% of the FLOPs and 7% of the cache.
- CSA compresses KV entries 4x and uses a FP4 lightning indexer for sparse block selection, while HCA compresses them 128x and applies dense attention over the compressed sequence.
- V4 preserves reasoning across user-message boundaries when tool calls are present, while non-tool conversations continue to flush reasoning at each new turn.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Rust](https://yomu.fyi/topic/rust)

- Source: [Hugging Face](https://huggingface.co/blog/deepseekv4)
- Source URL: https://huggingface.co/blog/deepseekv4
- Ingested by Yomu: 2026-08-29T01:48:16.780Z

[Read original post](https://huggingface.co/blog/deepseekv4)