REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff
Apple's new benchmark exposes a fundamental cliff in RL: most agents can't learn without resets, but the real world doesn't reset.
Why it matters
This research addresses a core limitation blocking deployment of autonomous systems in physical manipulation—the inability to train continuously without human intervention. The benchmark measures agent capability under irreversible conditions, a prerequisite for real-world robotics.
The key facts
10 to knowREVERSAL-BENCH: benchmark controlling reversibility via parameter ρ∈[0,1]
Targets continuous policy training without external resets
Tests eight manipulation settings across five physics engines
Includes reset oracle (ground-truth state recoverability verification)
Focus: gap between lab (reversible) and real-world (irreversible) manipulation
Published by Apple Machine Learning Research
Benchmark controls reversibility via continuous parameter ρ∈[0,1]
Includes reset oracle for ground-truth state recoverability verification
Addresses reset-free RL as a central goal for autonomous systems
From Apple Machine Learning Research
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: A central goal of autonomous reinforcement learning is continuous policy training without external resets. However, existing paradigms largely depend on underlying environmental reversibility, a property absent in real world manipulation, where events such as pushing objects off tables or spilling…