DeepSeek AI Releases DeepSeek-V4: Compressed Sparse Attention and Heavily Compressed Attention Enable One-Million-Token Contexts
One million tokens. DeepSeek-V4 just made long-context inference practical and cheap.
Why it matters
DeepSeek-V4 introduces compressed sparse attention mechanisms that enable 1M-token context windows at production-viable inference costs, challenging the scaling assumptions that have dominated model design. This is a capability leap that directly impacts how enterprises can use models for document reasoning, code repositories, and long-form synthesis.
The key facts
6 to knowDeepSeek-V4-Pro: 1.6T total parameters, 49B activated per token
DeepSeek-V4-Flash: 284B total parameters, 13B activated per token
Core innovation: Compressed Sparse Attention + Heavily Compressed Attention
One-million-token context window support
MoE architecture with inference cost optimization focus
Released as preview version
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: DeepSeek-AI has released a preview version of the DeepSeek-V4 series: two Mixture-of-Experts (MoE) language models built around one core challenge making one-million-token context windows practical and affordable at inference time. The series consists of DeepSeek-V4-Pro, with 1.6T total parameters…