ChipsThe story, in brief
Moonshot AI and Tsinghua Researchers Propose PrfaaS: A Cross-Datacenter KVCache Architecture that Rethinks How LLMs are Served at Scale
NOBODY TALKING: Everyone obsesses over model weights. Moonshot AI just rewrote the playbook for how those models actually run — across datacenters, at scale.
The KeyNews take
Why it matters
PrfaaS challenges the datacenter-bound architecture of LLM inference by decoupling prefill and decode across distributed infrastructure, potentially reshaping deployment economics and latency trade-offs for anyone running models at scale.
The key facts
6 to knowMoonshot AI + Tsinghua University collaboration
PrfaaS: cross-datacenter KVCache architecture
Breaks datacenter/rack confinement of RDMA-bound inference
Addresses prefill/decode co-location constraint
Infrastructure-layer innovation targeting LLM serving efficiency
Published April 2026 — recent research proposal
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: For years, the way large language models handle inference has been stuck inside a box — literally. The high-bandwidth RDMA networks that make modern LLM serving work have confined both prefill and decode to the same datacenter, sometimes even the same rack. A team of researchers at Moonshot AI and…