Robotic learning systems can appear capable when a simulator quietly resets every mistake. Apple researchers have introduced REVERSAL-BENCH to measure what happens when an agent must keep learning after actions that may be impossible to undo.
The benchmark controls reversibility with a continuous setting and uses a “reset oracle” to determine whether a state can still be recovered. It spans eight manipulation settings in five physics engines and evaluates standard actor-critic methods, safety-focused reinforcement learning and systems designed specifically for reset-free training.
Across those approaches, the researchers report a sharp failure cliff as tasks become less reversible. Reset-free agents eventually enter states from which they cannot recover, leaving further learning stuck. Agents trained in conventional episodes remain stable because an outside reset restores the environment. Comparisons with geometrically identical but reversible tasks indicate that irreversibility, rather than obstacle complexity alone, causes the breakdown.
The team also tested a safety shield designed to intervene before a permanent failure. It could predict recoverability accurately, but active recovery worked mainly when the agent still had room to steer away from the trap. The result highlights a practical limit for robots handling objects that can spill, fall or break: recognizing danger is useful only before the physical options disappear.