Teaching Theory of Mind to Robots to Enhance Collaboration

Nature is brimming with animals that collaborate in large numbers. Bees stake out the best feeding spots and let others know where they are. Ants construct complex hierarchical homes built for defense. Flocks of starlings move across the sky in beautiful formations as if they were a single entity.

None of these animals, however, collaborate in the way that humans do. Hive mind behaviors arise from simple rules followed by many individuals. Humans, however, have the ability to empathize with one another and predict each other’s actions—a trait known as Theory of Mind.

Now, a group of researchers from Duke University and Columbia University have figured out how to use this uniquely human trait to quickly train groups of robots to complete complex tasks. Whereas other control algorithms direct robots through mechanisms more similar to hive mind behaviors, this newly demonstrated framework called HUMAC teaches groups of robots how to collaborate through insights provided by a single human coach.

The research, accepted to the IEEE International Conference on Robotics and Automation (ICRA 2025), which will be held May 19-23, 2025, in Atlanta, Georgia, demonstrates how robots can learn to anticipate teammates’ actions, adapt strategies in real time and solve challenges that require human-like coordinated, collective intelligence.

The work could be a boon to applications such as wildfire response and wild survival tasks where robots need to cooperate and collaborate under constraints, with hierarchical team structures, uncertainty of the environment and communication bandwidth limits.

“Humans start to develop the skill of Theory of Mind around age four,” explained Boyuan Chen, the Dickinson Family Assistant Professor of Mechanical Engineering and Materials Science, Electrical and Computer Engineering, and Computer Science at Duke University. “It allows us to interpret and predict others’ intentions, allowing collaboration to emerge. This is an essential capability that our current robots are missing to allow them to work as a team with other robots and humans. We designed HUMAC to help robots learn from how humans think and coordinate in an efficient way.”

There have been other approaches to teaching robots to collaborate in meaningful tasks. One is to use reinforcement learning, where robots learn by interacting in the same environment with millions to billions of trials and errors, which is inefficient with no guarantee of success. Another method involves imitation learning from large groups of collaborative human experts, which is costly and impractical.

HUMAC takes a radically different approach. During training, the framework allows a single human operator to briefly take control of different robots within a team when necessary, guiding them at key strategic moments, much like a coach giving targeted advice during a soccer game. These interactions show the groups how to conduct sophisticated collaborative tactics like ambushing and encircling.

Following these brief demonstrations, the system embeds the human interventions into the robots’ algorithms. The key idea is that, for the robots to be able to learn to collaborate, they must learn to form a mental representation to simultaneously predict what their teammates’ plans are and what their opponent players will do. In other words, implicitly embedding all players’ decisions into their own plans—Theory of Mind.

“Our framework imagines the future of human-AI teaming where humans are leaders,” Chen said. “In this case, one human is guiding a larger number of agents in a fast and adaptable way, which has not been done before.”

The team tested HUMAC in a dynamic hide and seek game, where a team of three seeker robots try to catch a team of three faster-moving hider robots within a bounded square-shaped arena filled with random obstacles, relying solely on partial visual observations. This setting is challenging as non-collaborative seekers who keep chasing the closest hiders can only achieve a 36% success rate.

With HUMAC, a human coach selectively takes control of individual robots when necessary. After just 40 minutes of guidance, the robot team exhibits strong collaborative behaviors such as ambushing and encircling. In simulations, the success rate jumped to 84%, and even in physical ground vehicle tests, the success rate held strong at 80%.

“We observed robots starting to behave like genuine teammates,” said Zhengran Ji, the lead student author and graduate student in Chen’s lab. “They predicted each other’s movements and coordinated naturally, without explicit commands.”

“It was truly exciting to watch, and we believe it opens up many opportunities for future collaborative robot teams and human-robot teams in various applications,” Chen added.

Imagine a swarm of drones coordinating in real time to locate survivors after a natural disaster, efficiently sweeping through debris-covered areas without overlapping paths. Any application where a small number of humans need to teach a large number of robots how to collaborate could use this approach. Researchers are already working on expanding HUMAC to larger robot teams and more complex tasks while exploring richer interaction methods to streamline and enhance human-robot teaming.

“AI is not just a tool for humans, it’s a teammate. The final form of super-intelligence will not be AI alone nor humans alone, it’s the collective intelligence from both humans and AI,” Chen said. “Just as humans evolved to collaborate, AI will become more adaptive to work alongside with each other and with us. HUMAC is a step toward that future.”

This work is supported in part by ARL STRONG program under awards W911NF2320182 and W911NF2220113.

“Enabling Multi-Robot Collaboration from Single-Human Guidance.” Zhengran Ji, Lingyu Zhang, Paul Sajda, Boyuan Chen. IEEE International Conference on Robotics and Automation (ICRA 2025).