Global State Prediction for Collective Transport: From Implicit Communication to Memory-Enhanced Prediction for Decentralized Control
Abstract: This work investigates the application of deep reinforcement learning (DRL) to collective transport problems in multi-robot systems, with a particular focus on decentralized coordination and state prediction mechanisms. We begin by studying how different DRL algorithms perform in controlling aggregates of minimalistic robots that must transport objects without explicit communication, relying solely on implicit physical interactions. Building on these findings, we address the fundamental challenge of non-stationarity in multi-agent learning by developing novel approaches that eliminate the need for global information. We introduce Global State Prediction (GSP), a neural network architecture that enables robots to form beliefs about the swarm's state and predict its future behavior using only local information. We demonstrate that GSP enhances performance and robustness compared to methods requiring global knowledge, particularly in scenarios with unknown payload properties. Further advancing this framework, we develop memory-enhanced variants - Recurrent GSP and Attention GSP - which incorporate temporal dependencies and selective information processing. Through extensive ablation studies and real-world validation, we show that these predictive mechanisms improve performance across various scenarios. This work provides a comprehensive framework for decentralized multi-robot coordination, demonstrating how sophisticated prediction mechanisms can enable robust collective behavior using only local limited communication and without relying on global information.
Advisor: Professor Carlo Pinciroli
Committee: Professor Kevin Leahy, Professor Guanri Li, Professor Nicola Bezzo
Zoom Link: https://wpi.zoom.us/j/91321833265