Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration

For groups of autonomous agents to achieve a particular goal, they must engage in coordination and long-horizon reasoning.

Rather than relying on complex reward functions and explicit cooperation mechanisms, we ask what minimal ingredients are required for effective coordination and exploration to emerge in multi-agent settings.

We investigate this question through self-supervised goal-reaching, where agents aim to maximize the likelihood of visiting a goal state rather than maximizing a reward.

Despite a sparse feedback signal, we present empirical results that show self-supervised goal-reaching techniques enable agents to learn from such feedback.

On MARL benchmarks, self-supervised goal-reaching outperforms alternative approaches that have access to the same sparse reward signal.

Furthermore, we empirically demonstrate that multi-agent self-supervised goal-reaching approaches can be more robust than single-agent strategies.

While there is no explicit exploration mechanism, this approach explores nontrivial intermediate coordination strategies in sparse settings where alternative approaches fail to achieve a single success.