Apple researchers have proposed REFACTOR-VLA, a system that organizes a robot model’s raw actions into reusable motor programs. The goal is to address a weakness of many vision-language-action models: they issue short sequences of movements without learning clear skills that can be combined for longer tasks.
The system alternates between a “sleep” phase that groups action fragments with similar outcomes and a “wake” phase that uses a library of structured programs to generate actions. Candidate skills are kept only when they make the policy simpler without reducing its return. This structure is also meant to make learned behavior easier to inspect than a single monolithic action generator.
Tests used the LIBERO robot-manipulation benchmark. Simply increasing the learned world model from 188 million to 430 million parameters worsened performance on all four benchmark suites. By contrast, adding a supervised contrastive training objective improved how cleanly the system grouped skills. The result is research rather than a shipping Apple product, and benchmark performance does not yet establish reliability on physical robots in uncontrolled settings.