Reinforcement Learning in Robotics

Latest papers

• Learning Quantized Continuous Controllers for Integer Hardware

  Fabian Kresse, Christoph H. Lampert · 11 November 2025

  Deploying continuous-control reinforcement learning policies on embedded hardware requires meeting tight latency and power budgets. Small FPGAs can deliver these, but only if costly floating point pipelines are avoided. We study quantization-aware training (QAT) of policies for integer inference and…

• The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards

  Sukai Huang, Shu-Wei Liu, Nir Lipovetzky, Trevor Cohn · 11 November 2025

  While Vision-Language Models (VLMs) are increasingly used to generate reward signals for training embodied agents to follow instructions, our research reveals that agents guided by VLM rewards often underperform compared to those employing only intrinsic (exploration-driven) rewards, contradicting e…

• Rectifying Regression in Reinforcement Learning

  Alex Ayoub, David Szepesv\'ari, Alireza Bakhtiari, Csaba Szepesv\'ari, Dale Schuurmans · 11 November 2025

  This paper investigates the impact of the loss function in value-based methods for reinforcement learning through an analysis of underlying prediction objectives. We theoretically show that mean absolute error is a better prediction objective than the traditional mean squared error for controlling t…

• Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning

  Mingliang Zhang, Sichang Su, Chengyang He, Guillaume Sartoretti · 11 November 2025

  In multi-agent reinforcement learning (MARL), achieving multi-task generalization to diverse agents and objectives presents significant challenges. Existing online MARL algorithms primarily focus on single-task performance, but their lack of multi-task generalization capabilities typically results i…

• On The Presence of Double-Descent in Deep Reinforcement Learning

  Viktor Vesel\'y, Aleksandar Todorov, Matthia Sabatelli · 11 November 2025

  The double descent (DD) paradox, where over-parameterized models see generalization improve past the interpolation point, remains largely unexplored in the non-stationary domain of Deep Reinforcement Learning (DRL). We present preliminary evidence that DD exists in model-free DRL, investigating it s…

• Brain-Inspired Planning for Better Generalization in Reinforcement Learning

  Mingde "Harry" Zhao · 11 November 2025

  Existing Reinforcement Learning (RL) systems encounter significant challenges when applied to real-world scenarios, primarily due to poor generalization across environments that differ from their training conditions. This thesis explores the direction of enhancing agents' zero-shot systematic genera…

• Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search

  Samuel Sokota, Eugene Vinitsky, Hengyuan Hu, J. Zico Kolter, Gabriele Farina · 11 November 2025

  Few classical games have been regarded as such significant benchmarks of artificial intelligence as to have justified training costs in the millions of dollars. Among these, Stratego -- a board wargame exemplifying the challenge of strategic decision making under massive amounts of hidden informatio…

• Mirror Descent Policy Optimisation for Robust Constrained Markov Decision Processes

  David M. Bossens, Atsushi Nitanda · 11 November 2025

  Safety is an essential requirement for reinforcement learning systems. The newly emerging framework of robust constrained Markov decision processes allows learning policies that satisfy long-term constraints while providing guarantees under epistemic uncertainty. This paper presents mirror descent p…

• PADiff: Predictive and Adaptive Diffusion Policies for Ad Hoc Teamwork

  Hohei Chan, Xinzhi Zhang, Antao Xiang, Weinan Zhang, Mengchen Zhao · 11 November 2025

  Ad hoc teamwork (AHT) requires agents to collaborate with previously unseen teammates, which is crucial for many real-world applications. The core challenge of AHT is to develop an ego agent that can predict and adapt to unknown teammates on the fly. Conventional RL-based approaches optimize a singl…

• Learning to Focus: Prioritizing Informative Histories with Structured Attention Mechanisms in Partially Observable Reinforcement Learning

  Daniel De Dios Allegue, Jinke He, Frans A. Oliehoek · 11 November 2025

  Transformers have shown strong ability to model long-term dependencies and are increasingly adopted as world models in model-based reinforcement learning (RL) under partial observability. However, unlike natural language corpora, RL trajectories are sparse and reward-driven, making standard self-att…

• Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study

  Yongyu Mu, Jiali Zeng, Bei Li, Xinyan Guan, Fandong Meng, Jie Zhou, Tong Xiao, Jingbo Zhu · 11 November 2025

  Despite recent progress in training long-chain-of-thought reasoning models via scaling reinforcement learning (RL), its underlying training dynamics remain poorly understood, and several counterintuitive behaviors persist. This work focuses on three key aspects: (1) We systematically analyze the rol…

• Controllable Flow Matching for Online Reinforcement Learning

  Bin Wang, Boxiang Tao, Haifeng Jing, Hongbo Dou, Zijian Wang · 11 November 2025

  Model-based reinforcement learning (MBRL) typically relies on modeling environment dynamics for data efficiency. However, due to the accumulation of model errors over long-horizon rollouts, such methods often face challenges in maintaining modeling stability. To address this, we propose CtrlFlow, a …

• DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation

  Speed Zhu, Jianwei Cai, Guang Chen, Lulu Wu, Saiyong Yang, Wiggin Zhou · 11 November 2025

  Recent reasoning-first models (e.g., OpenAI o1, DeepSeek R1) have spurred a resurgence of interest in RLVR. Nevertheless, advances are dominated by mathematics (e.g., AIME), with competitive-programming code generation underexplored and data curation receiving less attention than RL algorithm design…

• MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios

  Xuantang Xiong, Ni Mu, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu · 11 November 2025

  Model-based reinforcement learning (MBRL) is a crucial approach to enhance the generalization capabilities and improve the sample efficiency of RL algorithms. However, current MBRL methods focus primarily on building world models for single tasks and rarely address generalization across different sc…

• Primal-Only Actor Critic Algorithm for Robust Constrained Average Cost MDPs

  Anirudh Satheesh, Sooraj Sathish, Swetha Ganesh, Keenan Powell, Vaneet Aggarwal · 11 November 2025

  In this work, we study the problem of finding robust and safe policies in Robust Constrained Average-Cost Markov Decision Processes (RCMDPs). A key challenge in this setting is the lack of strong duality, which prevents the direct use of standard primal-dual methods for constrained RL. Additional di…

• Distributionally Robust Self Paced Curriculum Reinforcement Learning

  Anirudh Satheesh, Keenan Powell, Vaneet Aggarwal · 11 November 2025

  A central challenge in reinforcement learning is that policies trained in controlled environments often fail under distribution shifts at deployment into real-world environments. Distributionally Robust Reinforcement Learning (DRRL) addresses this by optimizing for worst-case performance within an u…

• Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization

  Sayambhu Sen, Shalabh Bhatnagar · 11 November 2025

  Learning complex policies with Reinforcement Learning (RL) is often hindered by instability and slow convergence, a problem exacerbated by the difficulty of reward engineering. Imitation Learning (IL) from expert demonstrations bypasses this reliance on rewards. However, state-of-the-art IL methods,…

• Multi-agent Coordination via Flow Matching

  Dongsu Lee, Daehee Lee, Amy Zhang · 10 November 2025

  This work presents MAC-Flow, a simple yet expressive framework for multi-agent coordination. We argue that requirements of effective coordination are twofold: (i) a rich representation of the diverse joint behaviors present in offline data and (ii) the ability to act efficiently in real time. Howeve…

• Sample Complexity of Distributionally Robust Off-Dynamics Reinforcement Learning with Online Interaction

  Yiting He, Zhishuai Liu, Weixin Wang, Pan Xu · 10 November 2025

  Off-dynamics reinforcement learning (RL), where training and deployment transition dynamics are different, can be formulated as learning in a robust Markov decision process (RMDP) where uncertainties in transition dynamics are imposed. Existing literature mostly assumes access to generative models a…

• Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale

  Bassel Al Omari, Michael Matthews, Alexander Rutherford, Jakob Nicolaus Foerster · 10 November 2025

  Progress in multi-agent reinforcement learning (MARL) requires challenging benchmarks that assess the limits of current methods. However, existing benchmarks often target narrow short-horizon challenges that do not adequately stress the long-term dependencies and generalization capabilities inherent…

• Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward

  Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou · 10 November 2025

  Reinforcement Learning with Verifiable Rewards (RLVR) has propelled Large Language Models in complex reasoning, yet its scalability is often hindered by a training bottleneck where performance plateaus as policy entropy collapses, signaling a loss of exploration. Previous methods typically address t…

• Regret Lower Bounds for Decentralized Multi-Agent Stochastic Shortest Path Problems

  Utkarsh U. Chavan, Prashant Trivedi, Nandyala Hemachandra · 7 November 2025

  Multi-agent systems (MAS) are central to applications such as swarm robotics and traffic routing, where agents must coordinate in a decentralized manner to achieve a common objective. Stochastic Shortest Path (SSP) problems provide a natural framework for modeling decentralized control in such setti…

• Environment Agnostic Goal-Conditioning, A Study of Reward-Free Autonomous Learning

  Hampus {\AA}str\"om, Elin Anna Topp, Jacek Malec · 7 November 2025

  In this paper we study how transforming regular reinforcement learning environments into goal-conditioned environments can let agents learn to solve tasks autonomously and reward-free. We show that an agent can learn to solve tasks by selecting its own goals in an environment-agnostic way, at traini…

• The Peril of Preference: Why GRPO fails on Ordinal Rewards

  Anisha Garg, Ganesh Venkatesh · 7 November 2025

  Group-relative Policy Optimization's (GRPO) simplicity makes it highly desirable for adapting LLMs to become experts at specific tasks. But this simplicity also makes it ill-specified as we seek to enhance RL training with richer, non-binary feedback. When using ordinal rewards to give partial credi…

• Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data

  Lingkai Kong, Haichuan Wang, Tonghan Wang, Guojun Xiong, Milind Tambe · 7 November 2025

  Incorporating pre-collected offline data from a source environment can significantly improve the sample efficiency of reinforcement learning (RL), but this benefit is often challenged by discrepancies between the transition dynamics of the source and target environments. Existing methods typically a…