Abstract: Modern intelligent systems, such as social robots and autonomous vehicles, interact frequently with humans. Their behaviors are highly complex and dynamic, which results from unobservable social interactions. Thus, building reliable autonomy that safely navigates multi-agent scenarios requires scalable and generalizable relational reasoning and interaction modeling between interactive agents. Meanwhile, robots should be able to handle diverse and out-of-distribution (OOD) scenarios, which requires a robust and generalizable control policy. In this talk, I will introduce a unified relational reasoning framework for interaction modeling in multi-agent systems and human-robot interactions and its effectiveness in behavior prediction and sequential decision making. I will then introduce an effective approach to improve OOD generalization for autonomous navigation in interactive scenarios.