FrontierSeptember 17, 2026via Apple Machine Learning
REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff
Why it matters
This research addresses a core limitation blocking deployment of autonomous systems in physical manipulation—the inability to train continuously without human intervention. The benchmark measures agent capability under irreversible conditions, a prerequisite for real-world robotics.
Key signals
- REVERSAL-BENCH: benchmark controlling reversibility via parameter ρ∈[0,1]
- Targets continuous policy training without external resets
- Tests eight manipulation settings across five physics engines
- Includes reset oracle (ground-truth state recoverability verification)
- Focus: gap between lab (reversible) and real-world (irreversible) manipulation
- Published by Apple Machine Learning Research
- Benchmark controls reversibility via continuous parameter ρ∈[0,1]
- Includes reset oracle for ground-truth state recoverability verification
- Addresses reset-free RL as a central goal for autonomous systems
- From Apple Machine Learning Research
The hook
Apple's new benchmark exposes a fundamental cliff in RL: most agents can't learn without resets, but the real world doesn't reset.
A central goal of autonomous reinforcement learning is continuous policy training without external resets. However, existing paradigms largely depend on underlying environmental reversibility, a property absent in real world manipulation, where events such as pushing objects off tables or spilling g…