Reinforcement Learning in Robotics

Neueste Paper

• Statistical Guarantees for Offline Domain Randomization

  Arnaud Fickinger, Abderrahim Bendahi, Stuart Russell · 5. Februar 2026

  Reinforcement-learning (RL) agents often struggle when deployed from simulation to the real-world. A dominant strategy for reducing the sim-to-real gap is domain randomization (DR) which trains the policy across many simulators produced by sampling dynamics parameters, but standard DR ignores offlin…

• OAT: Ordered Action Tokenization

  Chaoqi Liu, Xiaoshen Han, Jiawei Gao, Yue Zhao, Haonan Chen, Yilun Du · 5. Februar 2026

  Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference. However, applying autoregressive modeling to continuous robot actions requires an effective action tokenization scheme. Existing approach…

• CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation

  Yannick Denker, Alexander Gepperth · 5. Februar 2026

  Continual reinforcement learning (CRL) requires agents to learn from a sequence of tasks without forgetting previously acquired policies. In this work, we introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator. Our Continual Robotic Simulation Suite…

• HoRD: Robust Humanoid Control via History-Conditioned Reinforcement Learning and Online Distillation

  Puyue Wang, Jiawei Hu, Yan Gao, Junyan Wang, Yu Zhang, Gillian Dobbie, Tao Gu, Wafa Johal, Ting Dang, Hong Jia · 5. Februar 2026

  Humanoid robots can suffer significant performance drops under small changes in dynamics, task specifications, or environment setup. We propose HoRD, a two-stage learning framework for robust humanoid control under domain shift. First, we train a high-performance teacher policy via history-condition…

• Rationality Measurement and Theory for Reinforcement Learning Agents

  Kejiang Qian, Amos Storkey, Fengxiang He · 5. Februar 2026

  This paper proposes a suite of rationality measures and associated theory for reinforcement learning agents, a property increasingly critical yet rarely explored. We define an action in deployment to be perfectly rational if it maximises the hidden true value function in the steepest direction. The …

• Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting

  Mehrdad Moghimi, Anthony Coache, Hyejin Ku · 5. Februar 2026

  Distributional reinforcement learning (RL) is a powerful framework increasingly adopted in safety-critical domains for its ability to optimize risk-sensitive objectives. However, the role of the discount factor is often overlooked, as it is typically treated as a fixed parameter of the Markov decisi…

• InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons

  Feiyu Gavin Zhu, Jean Oh, Reid Simmons · 5. Februar 2026

  Imitation learning has shown success in many tasks by learning from expert demonstrations. However, most existing work relies on large-scale demonstrations from technical professionals and close monitoring of the training process. These are challenging for a layperson when they want to teach the age…

• Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design

  Jaemoo Choi, Yuchen Zhu, Wei Guo, Petr Molodyk, Bo Yuan, Jinbin Bai, Yi Xin, Molei Tao, Yongxin Chen · 5. Februar 2026

  Reinforcement learning has been widely applied to diffusion and flow models for visual tasks such as text-to-image generation. However, these tasks remain challenging because diffusion models have intractable likelihoods, which creates a barrier for directly applying popular policy-gradient type met…

• SAFE: Stable Alignment Finetuning with Entropy-Aware Predictive Control for RLHF

  Dipan Maity · 5. Februar 2026

  Optimization (PPO) has been positioned by recent literature as the canonical method for the RL part of RLHF. PPO performs well empirically but has a heuristic motivation and handles the KL-divergence constraint used in LM-RLHF in an ad-hoc manner and suffers form reward oscillations, entropy collaps…

• Evolving Afferent Architectures: Biologically-inspired Models for Damage-Avoidance Learning

  Wolfgang Maass, Sabine Janzen, Prajvi Saxena, Sach Mukherjee · 5. Februar 2026

  We introduce Afferent Learning, a framework that produces Computational Afferent Traces (CATs) as adaptive, internal risk signals for damage-avoidance learning. Inspired by biological systems, the framework uses a two-level architecture: evolutionary optimization (outer loop) discovers afferent sens…

• Stochastic Decision Horizons for Constrained Reinforcement Learning

  Nikola Milosevic, Leonard Franz, Daniel Haeufle, Georg Martius, Nico Scherf, Pavel Kolev · 5. Februar 2026

  Constrained Markov decision processes (CMDPs) provide a principled model for handling constraints, such as safety and other auxiliary objectives, in reinforcement learning. The common approach of using additive-cost constraints and dual variables often hinders off-policy scalability. We propose a Co…

• On the Equilibrium between Feasible Zone and Uncertain Model in Safe Exploration

  Yujie Yang, Zhilong Zheng, Shengbo Eben Li · 5. Februar 2026

  Ensuring the safety of environmental exploration is a critical problem in reinforcement learning (RL). While limiting exploration to a feasible zone has become widely accepted as a way to ensure safety, key questions remain unresolved: what is the maximum feasible zone achievable through exploration…

• C-IDS: Solving Contextual POMDP via Information-Directed Objective

  Chongyang Shi, Michael Dorothy, Jie Fu · 5. Februar 2026

  We study the policy synthesis problem in contextual partially observable Markov decision processes (CPOMDPs), where the environment is governed by an unknown latent context that induces distinct POMDP dynamics. Our goal is to design a policy that simultaneously maximizes cumulative return and active…

• QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning

  Doyeon Lee, Eunyi Lyou, Hyunsoo Cho, Sookyung Kim, Joonseok Lee, Jaemoo Choi · 5. Februar 2026

  GRPO-style reinforcement learning (RL)-based LLM fine-tuning algorithms have recently gained popularity. Relying on heuristic trust-region approximations, however, they can lead to brittle optimization behavior, as global importance-ratio clipping and group-wise normalization fail to regulate sample…

• A Novel Framework for Uncertainty-Driven Adaptive Exploration

  Leonidas Bakopoulos, Georgios Chalkiadakis · 5. Februar 2026

  Adaptive exploration methods propose ways to learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that requ…

• DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory

  Wenxuan Zhou, Shujian Zhang, Brice Magdalou, John Lambert, Ehsan Amid, Richard Nock, Andrew Hard · 5. Februar 2026

  Normative theories allow one to elicit key parts of a ML algorithm from first principles, which is crucial at a time of championed scrutiny for ML work. Direct Preference Optimization (DPO) cleverly bypasses reward modeling by making an explicit link with a specific normative model of human choice. …

• Agent-Omit: Training Efficient LLM Agents for Adaptive Thought and Observation Omission via Agentic Reinforcement Learning

  Yansong Ning, Jun Fang, Naiqiang Tan, Hao Liu · 5. Februar 2026

  Managing agent thought and observation during multi-turn agent-environment interactions is an emerging strategy to improve agent efficiency. However, existing studies treat the entire interaction trajectories equally, overlooking the thought necessity and observation utility varies across turns. To …

• CRL-VLA: Continual Vision-Language-Action Learning

  Qixin Zeng, Shuo Zhang, Hongyin Zhang, Renjie Wang, Han Zhao, Libang Zhao, Runze Li, Donglin Wang, Chao Huang · 4. Februar 2026

  Lifelong learning is critical for embodied agents in open-world environments, where reinforcement learning fine-tuning has emerged as an important paradigm to enable Vision-Language-Action (VLA) models to master dexterous manipulation through environmental interaction. Thus, Continual Reinforcement …

• TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarking

  Yu Cheng, Jiuan Zhou, Yongkang Hu, Yihang Chen, Huichi Zhou, Mingang Chen, Zhizhong Zhang, Kun Shao, Yuan Xie, Zhaoxia Yin · 4. Februar 2026

  Test-time evolution of agent memory serves as a pivotal paradigm for achieving AGI by bolstering complex reasoning through experience accumulation. However, even during benign task evolution, agent safety alignment remains vulnerable-a phenomenon known as Agent Memory Misevolution. To evaluate this …

• Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity

  Yingxuan Yang, Chengrui Qu, Muning Wen, Laixi Shi, Ying Wen, Weinan Zhang, Adam Wierman, Shangding Gu · 4. Februar 2026

  LLM-based multi-agent systems (MAS) have emerged as a promising approach to tackle complex tasks that are difficult for individual LLMs. A natural strategy is to scale performance by increasing the number of agents; however, we find that such scaling exhibits strong diminishing returns in homogeneou…

• Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity

  Aneri Muni, Vincent Taboga, Esther Derman, Pierre-Luc Bacon, Erick Delage · 4. Februar 2026

  Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events. Unlike risk-neutral objectives, the static CVaR of the return depends on entire trajectories without admitting a recursive Bellman decomposition i…

• Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL

  Jinwoo Choi, Sang-Hyun Lee, Seung-Woo Seo · 4. Februar 2026

  Offline goal-conditioned reinforcement learning remains challenging for long-horizon tasks. While hierarchical approaches mitigate this issue by decomposing tasks, most existing methods rely on separate high- and low-level networks and generate only a single intermediate subgoal, making them inadequ…

• An Approximate Ascent Approach To Prove Convergence of PPO

  Leif Doering, Daniel Schmidt, Moritz Melcher, Sebastian Kassing, Benedikt Wille, Tilman Aach, Simon Weissmann · 4. Februar 2026

  Proximal Policy Optimization (PPO) is among the most widely used deep reinforcement learning algorithms, yet its theoretical foundations remain incomplete. Most importantly, convergence and understanding of fundamental PPO advantages remain widely open. Under standard theory assumptions we show how …

• Reparameterization Flow Policy Optimization

  Hai Zhong, Zhuoran Li, Xun Wang, Longbo Huang · 4. Februar 2026

  Reparameterization Policy Gradient (RPG) has emerged as a powerful paradigm for model-based reinforcement learning, enabling high sample efficiency by backpropagating gradients through differentiable dynamics. However, prior RPG approaches have been predominantly restricted to Gaussian policies, lim…

• Robust Intervention Learning from Emergency Stop Interventions

  Ethan Pronovost, Khimya Khetarpal, Siddhartha Srinivasa · 4. Februar 2026

  Human interventions are a common source of data in autonomous systems during testing. These interventions provide an important signal about where the current policy needs improvement, but are often noisy and incomplete. We define Robust Intervention Learning (RIL) as the problem of learning from int…