Teach-and-Repeat proposes a way to extract operational knowledge from mobile screen demonstrations for GUI agents.
The system is aimed at a practical bottleneck: agents need to learn how tasks are performed inside real interfaces, including sequences, constraints, and screen-specific cues.
If reliable, this kind of demonstration learning could make GUI agents easier to adapt to apps where written documentation is incomplete or nonexistent.