FrontierSeptember 13, 2026via MarkTechPost
A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth
Why it matters
A novel architecture approach to transformer scaling and context handling that challenges standard reset patterns at serving boundaries. Early-stage research with no empirical validation yet, but the design strategy (recurrent decoder state management, hardware-aware execution) could influence how inference systems handle long-context reasoning.
Key signals
- Recurrent Looped Transformer (RLT) — causal encoder + recurrent decoder
- Reference config: 48 encoder + 48 decoder layers = 96 logical blocks per token
- Decoder state path grows to 48t blocks after t tokens (unbounded temporal depth)
- Layerwise sliding-window attention cache carried across tokens with no serving-boundary reset
- Hardware-aware execution and RL replay contract specified
- No code, weights, or benchmark results released yet
- Yifan Zhang (Princeton) — technical report, not peer-reviewed publication
- Recurrent Looped Transformer (RLT) carries final hidden state and layerwise sliding-window attention cache across every token with no reset at serving boundary
- Configuration: 48 encoder + 48 decoder layers = 96 logical blocks per token
- State path grows to 48t decoder blocks after t tokens (unbounded temporal depth)
- Hardware-aware execution around recurrent core specified
- RL replay contract included in design
- No code, weights, or measured results released
- Princeton researcher Yifan Zhang; technical report only
The hook
Princeton researcher proposes RLT: a transformer that carries decoder state across every token, executing 96 logical blocks per token with unbounded temporal depth.
Yifan Zhang's Recurrent Looped Transformer (RLT) technical report proposes a causal encoder paired with a recurrent decoder that carries its final hidden state and layerwise sliding-window attention cache across every prompt and response token, with no reset at the serving boundary. The reference ti…