Recent progress in reinforcement learning (RL) has powered breakthroughs in various real-world problems (e.g., 1, 2, 3, 4, 5, 6), gathering considerable attention and investment. However, it has also exposed a significant gap between theoretical and experimental developments.

RL theory has grown significantly in the past two decades. Research has characterized the inherent difficulty of various settings, and has designed a wide variety of algorithms (e.g., 7, 8, 9) to reach optimal performances. Furthermore, a huge leap has been made in understanding how to handle large state spaces using function approximation techniques, identifying key structural properties that enable efficient learning (e.g., 10, 11, 12).

However, despite theoretical guarantees, applying RL algorithms to complex problems faces challenges. Theoretical algorithms often focus on simplified settings, making them hard to apply to real-world complexities. Furthermore, optimizing for worst-case scenarios, which include unlikely situations, can lead to algorithms that perform poorly on practical tasks. Yet, while specialized algorithms offer empirical success, they might not translate to other problems due to their specific design, and the reliance on heuristics and engineering fixes (13) further widens the gap between theory and practice.

With this workshop, we aim to bring theorists and experimentalists together to drive future research in RL.