Reinforcement Learning in Robotics

Latest papers

• Symmetry-Guided Memory Augmentation for Efficient Locomotion Learning

  Kaixi Bao, Chenhao Li, Yarden As, Andreas Krause, Marco Hutter · 11 February 2026

  Training reinforcement learning (RL) policies for legged locomotion often requires extensive environment interactions, which are costly and time-consuming. We propose Symmetry-Guided Memory Augmentation (SGMA), a framework that improves training efficiency by combining structured experience augmenta…

• CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning

  Xiaofeng Xiao, Xiao Hu, Yang Ye, Xubo Yue · 11 February 2026

  Reinforcement learning (RL) has achieved remarkable success in a wide range of sequential decision-making problems. Recent diffusion-based policies further improve RL by modeling complex, high-dimensional action distributions. However, existing diffusion policies primarily rely on statistical associ…

• ReflexGrad: A Dual-Process Architecture for Gradient-Free Inference-Time Learning

  Ankush Kadu, Ashwanth Krishnan · 11 February 2026

  Scaling inference-time compute has emerged as a powerful paradigm--yet deliberating longer is not the same as learning. Current approaches to extended reasoning in large language models allocate more computation to thinking but remain fundamentally static: they cannot adapt from mistakes encountered…

• ExO-PPO: an Extended Off-policy Proximal Policy Optimization Algorithm

  Hanyong Wang, Menglong Yang · 11 February 2026

  Deep reinforcement learning has been able to solve various tasks successfully, however, due to the construction of policy gradient and training dynamics, tuning deep reinforcement learning models remains challenging. As one of the most successful deep reinforcement-learning algorithm, the Proximal P…

• Latent Poincar\'e Shaping for Agentic Reinforcement Learning

  Hanchen Xia, Baoyou Chen, Zelin Zang, Yutang Ge, Guojiang Zhao, Siyu Zhu · 11 February 2026

  We propose LaPha, a method for training AlphaZero-like LLM agents in a Poincar\'e latent space. Under LaPha, the search process can be visualized as a tree rooted at the prompt and growing outward from the origin toward the boundary of the Poincar\'e ball, where negative curvature provides exponenti…

• A Controlled Study of Double DQN and Dueling DQN Under Cross-Environment Transfer

  Azka Nasir, Fatima Dossa, Muhammad Ahmed Atif, Mohammad Ahmed Atif · 11 February 2026

  Transfer learning in deep reinforcement learning is often motivated by improved stability and reduced training cost, but it can also fail under substantial domain shift. This paper presents a controlled empirical study examining how architectural differences between Double Deep Q-Networks (DDQN) and…

• Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning

  Nilaksh, Antoine Clavaud, Mathieu Reymond, Fran\c{c}ois Rivest, Sarath Chandar · 11 February 2026

  In streaming Reinforcement Learning (RL), transitions are observed and discarded immediately after a single update. While this minimizes resource usage for on-device applications, it makes agents notoriously sample-inefficient, since value-based losses alone struggle to extract meaningful representa…

• Optimistic World Models: Efficient Exploration in Model-Based Deep Reinforcement Learning

  Akshay Mete, Shahid Aamir Sheikh, Tzu-Hsiang Lin, Dileep Kalathil, P. R. Kumar · 11 February 2026

  Efficient exploration remains a central challenge in reinforcement learning (RL), particularly in sparse-reward environments. We introduce Optimistic World Models (OWMs), a principled and scalable framework for optimistic exploration that brings classical reward-biased maximum likelihood estimation …

• Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learning

  Zhida Jiang, Zhaolong Xing, Jiawei Lu, Yipei Niu, Qingyuan Sang, Liangxu Zhang, Wenquan Dai, Junhua Shu, Jiaxing Wang, Qiangyu Pei, Qiong Chen, Xinyu Liu, Fangming Liu, Ai Han, Zhen Chen, Ke Zhang · 11 February 2026

  Despite algorithm-level innovations for multi-agent reinforcement learning (MARL), the underlying networked infrastructure for large-scale MARL training remains underexplored. Existing training frameworks primarily optimize for single-agent scenarios and fail to address the unique system-level chall…

• Why Policy Gradient Algorithms Work for Undiscounted Total-Reward MDPs

  Jongmin Lee, Ernest K. Ryu · 11 February 2026

  The classical policy gradient method is the theoretical and conceptual foundation of modern policy-based reinforcement learning (RL) algorithms. Most rigorous analyses of such methods, particularly those establishing convergence guarantees, assume a discount factor $\gamma < 1$. In contrast, however…

• EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning

  Wujiang Xu, Wentian Zhao, Zhenting Wang, Yu-Jhe Li, Can Jin, Mingyu Jin, Kai Mei, Kun Wan, Dimitris N. Metaxas · 11 February 2026

  Training LLM agents in multi-turn environments with sparse rewards, where completing a single task requires 30+ turns of interaction within an episode, presents a fundamental challenge for reinforcement learning. We identify a critical failure mode unique to this setting: the exploration-exploitatio…

• On-Policy Policy Gradient Reinforcement Learning Without On-Policy Sampling

  Nicholas E. Corrado, Josiah P. Hanna · 11 February 2026

  On-policy reinforcement learning (RL) algorithms are typically characterized as algorithms that perform policy updates using i.i.d. trajectories collected by the agent's current policy. However, after observing only a finite number of trajectories, such on-policy sampling may produce data that fails…

• Risk-sensitive reinforcement learning using expectiles, shortfall risk and optimized certainty equivalent risk

  Sumedh Gupte, Shrey Rakeshkumar Patel, Soumen Pachal, Prashanth L. A., Sanjay P. Bhat · 11 February 2026

  We propose risk-sensitive reinforcement learning algorithms catering to three families of risk measures, namely expectiles, utility-based shortfall risk and optimized certainty equivalent risk. For each risk measure, in the context of a finite horizon Markov decision process, we first derive a polic…

• ADORA: Training Reasoning Models with Dynamic Advantage Estimation on Reinforcement Learning

  Qingnan Ren, Shiting Huang, Zhen Fang, Zehui Chen, Lin Chen, Lijun Li, Feng Zhao · 11 February 2026

  Reinforcement learning has become a cornerstone technique for developing reasoning models in complex tasks, ranging from mathematical problem-solving to imaginary reasoning. The optimization of these models typically relies on policy gradient methods, whose efficacy hinges on the accurate estimation…

• CODE-SHARP: Continuous Open-ended Discovery and Evolution of Skills as Hierarchical Reward Programs

  Richard Bornemann, Pierluigi Vito Amadori, Antoine Cully · 11 February 2026

  Developing agents capable of open-endedly discovering and learning novel skills is a grand challenge in Artificial Intelligence. While reinforcement learning offers a powerful framework for training agents to master complex skills, it typically relies on hand-designed reward functions. This is infea…

• Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning

  Jinzong Dong, Wei Huang, Jianshu Zhang, Zhuo Chen, Xinzhe Yuan, Qinying Gu, Zhaohui Jiang, Nanyang Ye · 10 February 2026

  Offline reinforcement learning (RL) optimizes policies from a previously collected static dataset and is an important branch of RL. A popular and promising approach is to regularize actor-critic methods with behavior cloning (BC), which yields realistic policies and mitigates bias from out-of-distri…

• On Randomness in Agentic Evals

  Bjarni Haukur Bjarnason, Andr\'e Silva, Martin Monperrus · 10 February 2026

  Agentic systems are evaluated on benchmarks where agents interact with environments to solve tasks. Most papers report a pass@1 score computed from a single run per task, assuming this gives a reliable performance estimate. We test this assumption by collecting 60,000 agentic trajectories on SWE-Ben…

• The Laplacian Keyboard: Beyond the Linear Span

  Siddarth Chandrasekar, Marlos C. Machado · 10 February 2026

  Across scientific disciplines, Laplacian eigenvectors serve as a fundamental basis for simplifying complex systems, from signal processing to quantum mechanics. In reinforcement learning (RL), these eigenvectors provide a natural basis for approximating reward functions; however, their use is typica…

• Efficient Anti-exploration via VQVAE and Fuzzy Clustering in Offline Reinforcement Learning

  Long Chen, Yinkui Liu, Shen Li, Bo Tang, Xuemin Hu · 10 February 2026

  Pseudo-count is an effective anti-exploration method in offline reinforcement learning (RL) by counting state-action pairs and imposing a large penalty on rare or unseen state-action pair data. Existing anti-exploration methods count continuous state-action pairs by discretizing these data, but ofte…

• Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions

  Yicheng Yang, Ruijiao Li, Lifeng Wang, Shuai Zheng, Shunzheng Ma, Keyu Zhang, Tuoyu Sun, Chenyun Dai, Jie Ding, Zhuo Zou · 10 February 2026

  This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a unified f…

• Regret Analysis of Unichain Average Reward Constrained MDPs with General Parameterization

  Anirudh Satheesh, Vaneet Aggarwal · 10 February 2026

  We study infinite-horizon average-reward constrained Markov decision processes (CMDPs) under the unichain assumption and general policy parameterizations. Existing regret analyses for constrained reinforcement learning largely rely on ergodicity or strong mixing-time assumptions, which fail to hold …

• Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning

  Constant Bourdrez, Alexandre V\'erine, Olivier Capp\'e · 10 February 2026

  Diffusion models generate samples through an iterative denoising process, guided by a neural network. While training the denoiser on real-world data is computationally demanding, the sampling procedure itself is more flexible. This adaptability serves as a key lever in practice, enabling improvement…

• Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learning

  David Hud\'ak, Maris F. L. Galesloot, Martin Tappler, Martin Kure\v{c}ka, Nils Jansen, Milan \v{C}e\v{s}ka · 10 February 2026

  Solving partially observable Markov decision processes (POMDPs) requires computing policies under imperfect state information. Despite recent advances, the scalability of existing POMDP solvers remains limited. Moreover, many settings require a policy that is robust across multiple POMDPs, further a…

• Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization

  Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma · 10 February 2026

  Multi-objective reinforcement learning (MORL) seeks to learn policies that balance multiple, often conflicting objectives. Although a single preference-conditioned policy is the most flexible and scalable solution, existing approaches remain brittle in practice, frequently failing to recover complet…

• Twice Sequential Monte Carlo for Tree Search

  Yaniv Oren, Joery A. de Vries, Pascal R. van der Vaart, Matthijs T. J. Spaan, Wendelin B\"ohmer · 10 February 2026

  Model-based reinforcement learning (RL) methods that leverage search are responsible for many milestone breakthroughs in RL. Sequential Monte Carlo (SMC) recently emerged as an alternative to the Monte Carlo Tree Search (MCTS) algorithm which drove these breakthroughs. SMC is easier to parallelize a…