Can robots learn to improvise like a child?

The company is building robots that learn by watching videos of humans performing chores, such as unzipping a purse or moving blocks. Instead of being programmed with a rigid set of rules for every specific movement, these robots act like apprentices. When faced with a new obstacle—like a missing tool—they show a surprising ability to improvise. In one demonstration, a robot used a dustpan to sweep up a block because its brush had been taken away. When it could not grip a set of banknotes, it switched to its other hand to find a better angle. While this sounds simple to a human, it is a significant shift from traditional robotics, which usually requires months of intense, task-specific coding.

Traditional robots rely on thousands of repetitive examples to learn one narrow chore. If the lighting changes or the object moves, the robot often gets lost. The team at Generalist is instead creating a general-purpose model, which acts like a brain that understands the physics of the world. To teach this brain, they have people wear special camera-equipped gloves to perform tasks, recording how a human hand navigates physical space. This data helps the robot understand relationships between objects—like how a hand needs to grip a handle to pull a zipper. By training the robot on this vast collection of human movements rather than just lines of code, the machine begins to develop a sense of cause and effect. It is not just copying a video; it is learning the underlying logic of manipulation.

We are still in the early days of this technology. These robots currently succeed at their assigned tasks about 59 percent of the time, which is far from the near-perfect reliability required for a factory floor or a home kitchen. However, the shift toward flexible, adaptive robots could eventually change how we handle tasks in manufacturing and beyond. If a robot can truly learn a new skill from a quick video rather than weeks of software engineering, we move closer to a world where machines can handle unexpected hurdles as easily as we do. It raises a simple question: if we can teach a machine to learn from its surroundings the way a toddler does, where will it stop experimenting?