Reinforcement Learning in Robotics

Latest papers

• A Reference Architecture of Reinforcement Learning Frameworks

  Xiaoran Liu, Istvan David · 9 March 2026

  The surge in reinforcement learning (RL) applications gave rise to diverse supporting technology, such as RL frameworks. However, the architectural patterns of these frameworks are inconsistent across implementations and there exists no reference architecture (RA) to form a common basis of compariso…

• The Coordination Gap: Alternation Metrics for Temporal Dynamics in Multi-Agent Battle of the Exes

  Nikolaos Al. Papadopoulos, Konstantinos Psannis · 9 March 2026

  Multi-agent coordination dilemmas expose a fundamental tension between individual optimization and collective welfare, yet characterizing such coordination requires metrics sensitive to temporal structure and collective dynamics. As a diagnostic testbed, we study a BoE-derived multi-agent variant of…

• One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning

  Yuan Pu, Yazhe Niu, Jia Tang, Junyu Xiong, Shuai Hu, Hongsheng Li · 9 March 2026

  In heterogeneous multi-task decision-making, tasks not only exhibit diverse observation and action spaces but also vary substantially in their underlying complexities. While conventional multi-task world models like UniZero excel in single-task settings, we find that when handling a broad and divers…

• Generative Predictive Control: Flow Matching Policies for Dynamic and Difficult-to-Demonstrate Tasks

  Vince Kurtz, Joel W. Burdick · 9 March 2026

  Generative control policies have recently unlocked major progress in robotics. These methods produce action sequences via diffusion or flow matching, with training data provided by demonstrations. But existing methods come with two key limitations: they require expert demonstrations, which can be di…

• Boosting deep Reinforcement Learning using pretraining with Logical Options

  Zihan Ye, Phil Chau, Raban Emunds, Jannis Bl\"uml, Cedric Derstroff, Quentin Delfosse, Oleg Arenz, Kristian Kersting · 9 March 2026

  Deep reinforcement learning agents are often misaligned, as they over-exploit early reward signals. Recently, several symbolic approaches have addressed these challenges by encoding sparse objectives along with aligned plans. However, purely symbolic architectures are complex to scale and difficult …

• Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments

  Michael Beukman, Khimya Khetarpal, Zeyu Zheng, Will Dabney, Jakob Foerster, Michael Dennis, Clare Lyle · 9 March 2026

  Plateaus, where an agent's performance stagnates at a suboptimal level, are a common problem in deep on-policy RL. Focusing on PPO due to its widespread adoption, we show that plateaus in certain regimes arise not because of known exploration, capacity, or optimization challenges, but because sample…

• TADPO: Reinforcement Learning Goes Off-road

  Zhouchonghao Wu, Raymond Song, Vedant Mundheda, Luis E. Navarro-Serment, Christof Schoenborn, Jeff Schneider · 9 March 2026

  Off-road autonomous driving poses significant challenges such as navigating unmapped, variable terrain with uncertain and diverse dynamics. Addressing these challenges requires effective long-horizon planning and adaptable control. Reinforcement Learning (RL) offers a promising solution by learning …

• Synthetic Monitoring Environments for Reinforcement Learning

  Leonard Pleiss, Carolin Schmidt, Maximilian Schiffer · 9 March 2026

  Reinforcement Learning (RL) lacks benchmarks that enable precise, white-box diagnostics of agent behavior. Current environments often entangle complexity factors and lack ground-truth optimality metrics, making it difficult to isolate why algorithms fail. We introduce Synthetic Monitoring Environmen…

• Latent Poincar\'e Shaping for Agentic Reinforcement Learning

  Hanchen Xia, Baoyou Chen, Zelin Zang, Yutang Ge, Guojiang Zhao, Siyu Zhu · 9 March 2026

  We propose LaPha, a method for training AlphaZero-like LLM agents in a Poincar\'e latent space. Under LaPha, the search process can be visualized as a tree rooted at the prompt and growing outward from the origin toward the boundary of the Poincar\'e ball, where negative curvature provides exponenti…

• Causally Robust Reward Learning from Reason-Augmented Preference Feedback

  Minjune Hwang, Yigit Korkmaz, Daniel Seita, Erdem B{\i}y{\i}k · 6 March 2026

  Preference-based reward learning is widely used for shaping agent behavior to match a user's preference, yet its sparse binary feedback makes it especially vulnerable to causal confusion. The learned reward often latches onto spurious features that merely co-occur with preferred trajectories during …

• TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback

  Lei Pang, Jun Luo, Ruinan Jin · 6 March 2026

  Group Relative Policy Optimization (GRPO), recently introduced by DeepSeek, is a critic-free reinforcement learning algorithm for fine-tuning large language models. GRPO replaces the value function in Proximal Policy Optimization (PPO) with group-normalized rewards while retaining PPO-style token-le…

• Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games

  Berkay Anahtarci, Can Deha Kariksiz, Naci Saldi · 6 March 2026

  We consider the maximum causal entropy inverse reinforcement learning (IRL) problem for infinite-horizon stationary mean-field games (MFG), in which we model the unknown reward function within a reproducing kernel Hilbert space (RKHS). This allows the inference of rich and potentially nonlinear rewa…

• Reward-Conditioned Reinforcement Learning

  Michal Nauman, Marek Cygan, Pieter Abbeel · 6 March 2026

  RL agents are typically trained under a single, fixed reward function, which makes them brittle to reward misspecification and limits their ability to adapt to changing task preferences. We introduce Reward-Conditioned Reinforcement Learning (RCRL), a framework that trains a single agent to optimize…

• CoRPO: Adding a Correctness Bias to GRPO Improves Generalization

  Anisha Garg, Claire Zhang, Nishit Neema, David Bick, Ganesh Venkatesh, Joel Hestness · 6 March 2026

  Group-Relative Policy Optimization (GRPO) has emerged as the standard for training reasoning capabilities in large language models through reinforcement learning. By estimating advantages using group-mean rewards rather than a learned critic, GRPO has enabled efficient scaling of reinforcement learn…

• Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics

  Kilian Freitag, Knut {\AA}kesson, Morteza Haghir Chehreghani · 6 March 2026

  Deep Reinforcement Learning is a promising tool for robotic control, yet practical application is often hindered by the difficulty of designing effective reward functions. Real-world tasks typically require optimizing multiple objectives simultaneously, necessitating precise tuning of their weights …

• EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection

  Shuo Yang, Soyeon Caren Han, Xueqi Ma, Yan Li, Mohammad Reza Ghasemi Madani, Eduard Hovy · 6 March 2026

  LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging due to delayed supervision and the difficulty of credit assignment in long-horizon trajectories. Existing optimization approaches tend to be either monolithic, which are p…

• Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning

  Ahmed Abouelazm, Tim Weinstein, Tim Joseph, Philip Sch\"orner, J. Marius Z\"ollner · 6 March 2026

  This paper addresses the challenges of training end-to-end autonomous driving agents using Reinforcement Learning (RL). RL agents are typically trained in a fixed set of scenarios and nominal behavior of surrounding road users in simulations, limiting their generalization and real-life deployment. W…

• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift

  Kevin Vogt-Lowell, Theodoros Tsiligkaridis, Rodney Lafuente-Mercado, Surabhi Ghatti, Shanghua Gao, Marinka Zitnik, Daniela Rus · 6 March 2026

  Real-world reinforcement learning systems must operate under distributional drift in their observation streams, yet most policy architectures implicitly assume fully observed and noise-free states. We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures …

• VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory

  Yuheng Lei, Zhixuan Liang, Hongyuan Zhang, Ping Luo · 6 March 2026

  Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory. Simply enlarging the con…

• Complexity-Regularized Proximal Policy Optimization

  Luca Serfilippi, Giorgio Franceschelli, Antonio Corradi, Mirco Musolesi · 6 March 2026

  Policy gradient methods usually rely on entropy regularization to prevent premature convergence. However, maximizing entropy indiscriminately pushes the policy towards a uniform distribution, often overriding the reward signal if not optimally tuned. We propose replacing the standard entropy term wi…

• RoboPocket: Improve Robot Policies Instantly with Your Phone

  Junjie Fang, Wendi Chen, Han Xue, Fangyuan Zhou, Tian Le, Yi Wang, Yuting Zhang, Jun Lv, Chuan Wen, Cewu Lu · 6 March 2026

  Scaling imitation learning is fundamentally constrained by the efficiency of data collection. While handheld interfaces have emerged as a scalable solution for in-the-wild data acquisition, they predominantly operate in an open-loop manner: operators blindly collect demonstrations without knowing th…

• Diffusion Policy through Conditional Proximal Policy Optimization

  Ben Liu, Shunpeng Yang, Hua Chen · 6 March 2026

  Reinforcement learning (RL) has been extensively employed in a wide range of decision-making problems, such as games and robotics. Recently, diffusion policies have shown strong potential in modeling multi-modal behaviors, enabling more diverse and flexible action generation compared to the conventi…

• CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions

  Lizhi Yang, Blake Werner, Massimiliano de Sa, Aaron D. Ames · 6 March 2026

  Reinforcement learning (RL), while powerful and expressive, can often prioritize performance at the expense of safety. Yet safety violations can lead to catastrophic outcomes in real-world deployments. Control Barrier Functions (CBFs) offer a principled method to enforce dynamic safety -- traditiona…

• SCoUT: Scalable Communication via Utility-Guided Temporal Grouping in Multi-Agent Reinforcement Learning

  Manav Vora, Gokul Puthumanaillam, Hiroyasu Tsukamoto, Melkior Ornik · 6 March 2026

  Communication can improve coordination in partially observed multi-agent reinforcement learning (MARL), but learning \emph{when} and \emph{who} to communicate with requires choosing among many possible sender-recipient pairs, and the effect of any single message on future reward is hard to isolate. …

• Latent Policy Steering through One-Step Flow Policies

  Hokyun Im, Andrey Kolobov, Jianlong Fu, Youngwoon Lee · 6 March 2026

  Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration. Yet, offline RL's performance often hinges on a brittle trade-off between (1) return maximization, which can push policies outside the dataset support, and (2) behavioral constraints, which ty…