Reinforcement Learning in Robotics

Latest papers

• Statistical Reinforcement Learning in the Real World: A Survey of Challenges and Future Directions

  Asim H. Gazi, Yongyi Guo, Daiqi Gao, Ziping Xu, Kelly W. Zhang, Susan A. Murphy · 23 January 2026

  Reinforcement learning (RL) has achieved remarkable success in real-world decision-making across diverse domains, including gaming, robotics, online advertising, public health, and natural language processing. Despite these advances, a substantial gap remains between RL research and its deployment i…

• Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors

  Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong · 23 January 2026

  Large language models (LLMs) can call tools effectively, yet they remain brittle in multi-turn execution: following a tool call error, smaller models often degenerate into repetitive invalid re-invocations, failing to interpret error feedback and self-correct. This brittleness hinders reliable real-…

• Representation-Driven Reinforcement Learning

  Ofir Nabati, Guy Tennenholtz, Shie Mannor · 23 January 2026

  We present a representation-driven framework for reinforcement learning. By representing policies as estimates of their expected values, we leverage techniques from contextual bandits to guide exploration and exploitation. Particularly, embedding a policy network into a linear feature space allows u…

• PROMA: Projected Microbatch Accumulation for Reference-Free Proximal Policy Updates

  Nilin Abrahamsen · 23 January 2026

  This note introduces Projected Microbatch Accumulation (PROMA), a proximal policy method that modifies gradient accumulation across microbatches rather than relying on likelihood ratios relative to a reference policy. During accumulation, PROMA projects the partially accumulated gradient to be ortho…

• Proximal Policy Optimization with Evolutionary Mutations

  Casimir Czworkowski, Stephen Hornish, Alhassan S. Yasin · 22 January 2026

  Proximal Policy Optimization (PPO) is a widely used reinforcement learning algorithm known for its stability and sample efficiency, but it often suffers from premature convergence due to limited exploration. In this paper, we propose POEM (Proximal Policy Optimization with Evolutionary Mutations), a…

• Rethinking Reinforcement fine-tuning of LLMs: A Multi-armed Bandit Learning Perspective

  Xiao Hu, Hong Xie, Tao Tan, Defu Lian, Jianyu Han · 22 January 2026

  A large number of heuristics have been proposed to optimize the reinforcement fine-tuning of LLMs. However, inconsistent claims are made from time to time, making this area elusive. Reflecting on this situation, two fundamental questions still lack a clear understanding: 1) what is the role of each …

• Case-Guided Sequential Assay Planning in Drug Discovery

  Tianchi Chen, Jan Bima, Sean L. Wu, Otto Ritter, Bingjia Yang, Xiang Yu · 22 January 2026

  Optimally sequencing experimental assays in drug discovery is a high-stakes planning problem under severe uncertainty and resource constraints. A primary obstacle for standard reinforcement learning (RL) is the absence of an explicit environment simulator or transition data $(s, a, s')$; planning mu…

• Constrained Black-Box Attacks Against Cooperative Multi-Agent Reinforcement Learning

  Amine Andam, Jamal Bentahar, Mustapha Hedabou · 22 January 2026

  Collaborative multi-agent reinforcement learning has rapidly evolved, offering state-of-the-art algorithms for real-world applications, including sensitive domains. However, a key challenge to its widespread adoption is the lack of a thorough investigation into its vulnerabilities to adversarial att…

• Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning

  Oleg Shchendrigin, Egor Cherepanov, Alexey K. Kovalev, Aleksandr I. Panov · 22 January 2026

  Effective decision-making in the real world depends on memory that is both stable and adaptive: environments change over time, and agents must retain relevant information over long horizons while also updating or overwriting outdated content when circumstances shift. Existing Reinforcement Learning …

• Improving Regret Approximation for Unsupervised Dynamic Environment Generation

  Harry Mead, Bruno Lacerda, Jakob Foerster, Nick Hawes · 22 January 2026

  Unsupervised Environment Design (UED) seeks to automatically generate training curricula for reinforcement learning (RL) agents, with the goal of improving generalisation and zero-shot performance. However, designing effective curricula remains a difficult problem, particularly in settings where sma…

• Shaping Sparse Rewards in Reinforcement Learning: A Semi-supervised Approach

  Wenyun Li, Wenjie Huang, Chen Sun · 22 January 2026

  In many real-world scenarios, reward signal for agents are exceedingly sparse, making it challenging to learn an effective reward function for reward shaping. To address this issue, the proposed approach in this paper performs reward shaping not only by utilizing non-zero-reward transitions but also…

• Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection

  Jennifer Dodgson, Alfath Daryl Alhajir, Michael Joedhitya, Akira Rafhael Janson Pattirane, Surender Suresh Kumar, Joseph Lim, C. H. Peh, Adith Ramdas, Steven Zhang Zhexu · 21 January 2026

  Self-training systems often degenerate due to the lack of an external criterion for judging data quality, leading to reward hacking and semantic drift. This paper provides a proof-of-concept system architecture for stable self-training under sparse external feedback and bounded memory, and empirical…

• From Relative Entropy to Minimax: A Unified Framework for Coverage in MDPs

  Xihe Gu, Urbashi Mitra, Tara Javidi · 21 January 2026

  Targeted and deliberate exploration of state--action pairs is essential in reward-free Markov Decision Problems (MDPs). More precisely, different state-action pairs exhibit different degree of importance or difficulty which must be actively and explicitly built into a controlled exploration strategy…

• GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning

  Shutong Ding, Ke Hu, Shan Zhong, Haoyang Luo, Weinan Zhang, Jingya Wang, Jun Wang, Ye Shi · 21 January 2026

  Recent advances in reinforcement learning (RL) have demonstrated the powerful exploration capabilities and multimodality of generative diffusion-based policies. While substantial progress has been made in offline RL and off-policy RL settings, integrating diffusion policies into on-policy frameworks…

• Continual Knowledge Adaptation for Reinforcement Learning

  Jinwu Hu, Zihao Lian, Zhiquan Wen, Chenghao Li, Guohao Chen, Xutao Wen, Bin Xiao, Mingkui Tan · 21 January 2026

  Reinforcement Learning enables agents to learn optimal behaviors through interactions with environments. However, real-world environments are typically non-stationary, requiring agents to continuously adapt to new tasks and changing conditions. Although Continual Reinforcement Learning facilitates l…

• Controlling Underestimation Bias in Constrained Reinforcement Learning for Safe Exploration

  Shiqing Gao, Jiaxin Ding, Luoyi Fu, Xinbing Wang · 21 January 2026

  Constrained Reinforcement Learning (CRL) aims to maximize cumulative rewards while satisfying constraints. However, existing CRL algorithms often encounter significant constraint violations during training, limiting their applicability in safety-critical scenarios. In this paper, we identify the und…

• Extreme Value Policy Optimization for Safe Reinforcement Learning

  Shiqing Gao, Yihang Zhou, Shuai Shao, Haoyu Luo, Yiheng Bing, Jiaxin Ding, Luoyi Fu, Xinbing Wang · 21 January 2026

  Ensuring safety is a critical challenge in applying Reinforcement Learning (RL) to real-world scenarios. Constrained Reinforcement Learning (CRL) addresses this by maximizing returns under predefined constraints, typically formulated as the expected cumulative cost. However, expectation-based constr…

• Rethinking Entropy Interventions in RLVR: An Entropy Change Perspective

  Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, Jiawei Chen · 21 January 2026

  While Reinforcement Learning with Verifiable Rewards (RLVR) can enhance LLM reasoning, its training process carries a critical risk: entropy collapse. This phenomenon is a rapid decrease in policy entropy, which severely limits exploration and diminishes learning effectiveness. Recent methods attemp…

• Group-Invariant Unsupervised Skill Discovery: Symmetry-aware Skill Representations for Generalizable Behavior

  Junwoo Chang, Joseph Park, Roberto Horowitz, Jongmin Lee, Jongeun Choi · 21 January 2026

  Unsupervised skill discovery aims to acquire behavior primitives that improve exploration and accelerate downstream task learning. However, existing approaches often ignore the geometric symmetries of physical environments, leading to redundant behaviors and sample inefficiency. To address this, we …

• Hindsight Preference Replay Improves Preference-Conditioned Multi-Objective Reinforcement Learning

  Jonaid Shianifar, Michael Schukat, Karl Mason · 21 January 2026

  Multi-objective reinforcement learning (MORL) enables agents to optimize vector-valued rewards while respecting user preferences. CAPQL, a preference-conditioned actor-critic method, achieves this by conditioning on weight vectors w and restricts data usage to the specific preferences under which it…

• Vejde: A Framework for Inductive Deep Reinforcement Learning Based on Factor Graph Color Refinement

  Jakob Nyberg, Pontus Johnson · 21 January 2026

  We present and evaluate Vejde; a framework which combines data abstraction, graph neural networks and reinforcement learning to produce inductive policy functions for decision problems with richly structured states, such as object classes and relations. MDP states are represented as data bases of fa…

• Evaluation-Aware Reinforcement Learning

  Shripad Vilasrao Deshmukh, Will Schwarzer, Scott Niekum · 21 January 2026

  Policy evaluation is often a prerequisite for deploying safety- and performance-critical systems. Existing evaluation approaches frequently suffer from high variance due to limited data and long-horizon tasks, or high bias due to unequal support or inaccurate environmental models. We posit that thes…

• Distribution-Centric Policy Optimization Dominates Exploration-Exploitation Trade-off

  Zhaochun Li, Chen Wang, Jionghao Bai, Shisheng Cui, Ge Lan, Zhou Zhao, Yue Wang · 21 January 2026

  The exploration-exploitation (EE) trade-off is a central challenge in reinforcement learning (RL) for large language models (LLMs). With Group Relative Policy Optimization (GRPO), training tends to be exploitation driven: entropy decreases monotonically, samples convergence, and exploration fades. M…

• Communication Methods in Multi-Agent Reinforcement Learning

  Christoph Wittner · 21 January 2026

  Multi-agent reinforcement learning is a promising research area that extends established reinforcement learning approaches to problems formulated as multi-agent systems. Recently, a multitude of communication methods have been introduced to this field to address problems such as partially observable…

• R$^2$PO: Decoupling Training Trajectories from Inference Responses for LLM Reasoning

  Jingchu Wang, Bingbing Xu, Yige Yuan, Bin Xie, Xiaoqian Sun, Huawei Shen · 21 January 2026

  Reinforcement learning has become a central paradigm for improving LLM reasoning. However, existing methods use a single policy to produce both inference responses and training optimization trajectories. The objective conflict between generating stable inference responses and diverse training trajec…