End-To-End Imitation Learning for Socially Aware Navigation using Memory-based Trajectory and Action Planning

The increasing deployment of service robots and autonomous vehicles operating in human-populated environments underscores the importance of planning trajectories to ensure pedestrian comfort while optimizing robot travel times. End-to-end models are promising for shortcutting the complex problem of pedestrian detection, trajectory prediction, and robot movement planning by using a single network. Imitation Learning (IL) has been widely adopted in self-driving because it enables implicit understanding of complex environments and expert driving behaviour, whereas socially aware navigation has usually relied on Reinforcement Learning (RL) approaches that learn from abstracted environment representations, prioritizing information relevant to human interaction. In this research, we present Memory-based Trajectory and Action Planning (MTAP), an IL approach extended for socially aware navigation to perform intelligent pedestrian avoidance while maintaining a deeper understanding of the environment and the expert driving behaviour. Our method predicts the trajectory and the control action commands directly from the information of the sensors and the global target, employing a recurrent layer to anticipate pedestrian movements based on past frames. The network is trained and evaluated in the CARLA simulator, where we introduce a benchmark for assessing pedestrian avoidance efficacy, presenting also new social navigation metrics to measure both trajectory efficiency and pedestrian comfort during avoidance maneuvers.