A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth
Princeton researcher proposes RLT: a transformer that carries decoder state across every token, executing 96 logical blocks per token with unbounded temporal depth.
Why it matters
A novel architecture approach to transformer scaling and context handling that challenges standard reset patterns at serving boundaries. Early-stage research with no empirical validation yet, but the design strategy (recurrent decoder state management, hardware-aware execution) could influence how inference systems handle long-context reasoning.
The key facts
14 to knowRecurrent Looped Transformer (RLT) — causal encoder + recurrent decoder
Reference config: 48 encoder + 48 decoder layers = 96 logical blocks per token
Decoder state path grows to 48t blocks after t tokens (unbounded temporal depth)
Layerwise sliding-window attention cache carried across tokens with no serving-boundary reset
Hardware-aware execution and RL replay contract specified
No code, weights, or benchmark results released yet
Yifan Zhang (Princeton) — technical report, not peer-reviewed publication
Recurrent Looped Transformer (RLT) carries final hidden state and layerwise sliding-window attention cache across every token with no reset at serving boundary
Configuration: 48 encoder + 48 decoder layers = 96 logical blocks per token
State path grows to 48t decoder blocks after t tokens (unbounded temporal depth)
Hardware-aware execution around recurrent core specified
RL replay contract included in design
No code, weights, or measured results released
Princeton researcher Yifan Zhang; technical report only
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Yifan Zhang's Recurrent Looped Transformer (RLT) technical report proposes a causal encoder paired with a recurrent decoder that carries its final hidden state and layerwise sliding-window attention cache across every prompt and response token, with no reset at the serving boundary. The reference…