Naila Sebastián Esandi

Zero-shot reinforcement learning (ZSRL) represents a critical frontier in artificial intelligence, aiming to develop agents capable of solving novel tasks in unseen environments without requiring additional training. However, the current theoretical understanding of generalization in this domain is limited, with a notable absence of rigorous, problem-specific generalization bounds. While empirical or practical advancements have been made through techniques like Diversity Is All You Need (DIAYN) and the use of feature-based (FB) representations, most methods often lack a formal framework for quantifying their out-of-distribution performance guarantees. This thesis will seek to bridge this gap by first exploring the problem of zero-shot bandits as a foundational starting point. Building upon this, it will derive novel, tighter generalization bounds for a broad class of ZSRL algorithms by extending existing PAC-Bayesian and Rademacher complexity frameworks. Based on these theoretical insights, we will propose an optimized information-theoretic exploration method designed to actively learn the successor representation of the environment during pre-training, moving beyond the implicit diversity encouraged by prior methods. We will aim to demonstrate that our method's structured exploration leads to a robust representation of the environment's dynamics, which should directly translate to improved theoretical and empirical zero-shot transfer performance. Ultimately, the work presented herein could provide a foundational theoretical framework for analyzing ZSRL agents while also contributing a practical, state-of-the-art algorithm that will tighten the gap between training and test error, thereby advancing the field towards more reliable and generalizable agents.