In October 2020 Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter published “Learning quadrupedal locomotion over challenging terrain” in Science Robotics. The work trained a single neural-network controller for the four-legged ANYmal robot and showed it walking robustly through natural environments it had never encountered, including deformable mud and snow, shifting rubble, thick vegetation, and gushing water.
The striking part is that the controller is “blind.” It uses only proprioception, the stream of joint angles, velocities, and torques the robot can feel in its own body, with no cameras or external sensors. Conventional legged controllers relied on elaborate hand-tuned state machines that explicitly trigger motion primitives; this controller instead learned a single policy in simulation that generalized to the real world with zero fine-tuning on the physical robot.
The training used a teacher-student scheme: a teacher policy first learned with access to privileged information about the terrain in simulation, then a student policy was trained to reproduce that behavior using only the proprioceptive signals available on the real machine. The authors argued that radical robustness in messy natural terrain can come from training in much simpler simulated domains.
For a general reader, this paper is a milestone in making legged robots actually useful outdoors, where the ground is uneven, slippery, and unpredictable, and where a robot cannot count on seeing every foothold.