Reinforcement Learning in Robotics

Neueste Paper

• ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

  Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Pengyong Wang, Ailin Ren, Xin Li, Pengjun Xie, Jiawei Liu, Ning Guo, Jingren Zhou, Zheng-Jun Zha · 13. Januar 2026

  Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on open-ended agent tasks with vast solution spaces (e.g., complex travel planning). Due to the absence of objective ground-truth for these tasks, current RL algo…

• A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Control

  Wonhyeok Choi, Minwoo Choi, Jungwan Woo, Kyumin Hwang, Jaeyeul Kim, Sunghoon Im · 13. Januar 2026

  Diffusion policies have emerged as a powerful approach for robotic control, demonstrating superior expressiveness in modeling multimodal action distributions compared to conventional policy networks. However, their integration with online reinforcement learning remains challenging due to fundamental…

• On the Non-decoupling of Supervised Fine-tuning and Reinforcement Learning in Post-training

  Xueyan Niu, Bo Bai, Wei Han, Weixi Zhang · 13. Januar 2026

  Post-training of large language models routinely interleaves supervised fine-tuning (SFT) with reinforcement learning (RL). These two methods have different objectives: SFT minimizes the cross-entropy loss between model outputs and expert responses, while RL maximizes reward signals derived from hum…

• No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learning

  Zhicong Li, Lingjie Jiang, Yulan Hu, Xingchen Zeng, Yixia Li, Xiangwen Zhang, Guanhua Chen, Zheng Pan, Xin Li, Yong Liu · 13. Januar 2026

  Critique-guided reinforcement learning (RL) has emerged as a powerful paradigm for training LLM agents by augmenting sparse outcome rewards with natural-language feedback. However, current methods often rely on static or offline critic models, which fail to adapt as the policy evolves. In on-policy …

• Stagewise Reinforcement Learning and the Geometry of the Regret Landscape

  Chris Elliott, Einar Urdshals, David Quarel, Matthew Farrugia-Roberts, Daniel Murfet · 13. Januar 2026

  Singular learning theory characterizes Bayesian learning as an evolving tradeoff between accuracy and complexity, with transitions between qualitatively different solutions as sample size increases. We extend this theory to deep reinforcement learning, proving that the concentration of the generaliz…

• Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulation

  Huanyu Li, Kun Lei, Sheng Zang, Kaizhe Hu, Yongyuan Liang, Bo An, Xiaoli Li, Huazhe Xu · 13. Januar 2026

  Post-training algorithms based on deep reinforcement learning can push the limits of robotic models for specific objectives, such as generalizability, accuracy, and robustness. However, Intervention-requiring Failures (IR Failures) (e.g., a robot spilling water or breaking fragile glass) during real…

• OpenTinker: Separating Concerns in Agentic Reinforcement Learning

  Siqi Zhu, Jiaxuan You · 13. Januar 2026

  We introduce OpenTinker, an infrastructure for reinforcement learning (RL) of large language model (LLM) agents built around a separation of concerns across algorithm design, execution, and agent-environment interaction. Rather than relying on monolithic, end-to-end RL pipelines, OpenTinker decompos…

• Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training

  Xue Gong, Qi Yi, Ziyuan Nan, Guanhua Huang, Kejiao Li, Yuhao Jiang, Ruibin Xiong, Zenan Xu, Jiaming Guo, Shaohui Peng, Bo Zhou · 13. Januar 2026

  Training Large Language Models (LLMs) for reasoning tasks is increasingly driven by Reinforcement Learning with Verifiable Rewards (RLVR), where Proximal Policy Optimization (PPO) provides a principled framework for stable policy updates. However, the practical application of PPO is hindered by unre…

• Offline Meta-Reinforcement Learning with Flow-Based Task Inference and Adaptive Correction of Feature Overgeneralization

  Min Wang, Xin Li, Mingzhong Wang, Hasnaa Bennis · 13. Januar 2026

  Offline meta-reinforcement learning (OMRL) combines the strengths of learning from diverse datasets in offline RL with the adaptability to new tasks of meta-RL, promising safe and efficient knowledge acquisition by RL agents. However, OMRL still suffers extrapolation errors due to out-of-distributio…

• FlowRL: Flow-Augmented Few-Shot Reinforcement Learning for Semi-Structured Sensor Data

  Mohammad Pivezhandi, Abusayeed Saifullah · 13. Januar 2026

  Reinforcement learning (RL) in few-shot scenarios with limited sensor data is challenging due to insufficient training samples, particularly in applications like Dynamic Voltage and Frequency Scaling (DVFS) where sensor readings are semi-structured with inherent correlations. We propose Flow-Augment…

• Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning

  Ziheng Li, Liu Kang, Feng Xiao, Luxi Xing, Qingyi Si, Zhuoran Li, Weikang Gong, Deqing Yang, Yanghua Xiao, Hongcheng Guo · 13. Januar 2026

  Group Relative Policy Optimization (GRPO) has emerged as a promising critic-free reinforcement learning paradigm for reasoning tasks. However, standard GRPO employs a coarse-grained credit assignment mechanism that propagates group-level rewards uniformly to to every token in a sequence, neglecting …

• Optimal Learning Rate Schedule for Balancing Effort and Performance

  Valentina Njaradi, Rodrigo Carrasco-Davis, Peter E. Latham, Andrew Saxe · 13. Januar 2026

  Learning how to learn efficiently is a fundamental challenge for biological agents and a growing concern for artificial ones. To learn effectively, an agent must regulate its learning speed, balancing the benefits of rapid improvement against the costs of effort, instability, or resource use. We int…

• ToolBrain: A Flexible Reinforcement Learning Framework for Agentic Tools

  Quy Minh Le, Minh Sao Khue Luu, Khanh-Tung Tran, Duc-Hai Nguyen, Hoang-Quoc-Viet Pham, Quan Le, Hoang Thanh Lam, Hoang D. Nguyen · 13. Januar 2026

  Effective tool use is essential for agentic AI, yet training agents to utilize tools remains challenging due to manually designed rewards, limited training data, and poor multi-tool selection, resulting in slow adaptation, wasted computational resources, and suboptimal performance. We introduce Tool…

• Object-Centric World Models Meet Monte Carlo Tree Search

  Rodion Vakhitov, Leonid Ugadiarov, Aleksandr Panov · 13. Januar 2026

  In this paper, we introduce ObjectZero, a novel reinforcement learning (RL) algorithm that leverages the power of object-level representations to model dynamic environments more effectively. Unlike traditional approaches that process the world as a single undifferentiated input, our method employs G…

• Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Search

  Ping Guo, Chao Li, Yinglan Feng, Chaoning Zhang · 13. Januar 2026

  Designing effective control policies for autonomous systems remains a fundamental challenge, traditionally addressed through reinforcement learning or manual engineering. While reinforcement learning has achieved remarkable success, it often suffers from high sample complexity, reward shaping diffic…

• Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration

  Yang Zhao, Yangou Ouyang, Xiao Ding, Hepeng Wang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu · 13. Januar 2026

  While Hybrid Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) has become the standard paradigm for training LLM agents, effective mechanisms for data allocation between these stages remain largely underexplored. Current data arbitration strategies often rely on surface-level heur…

• Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learning

  Sijia li, Xinran Li, Shibo Chen, Jun Zhang · 13. Januar 2026

  Offline multi-agent reinforcement learning (MARL) aims to solve cooperative decision-making problems in multi-agent systems using pre-collected datasets. Existing offline MARL methods primarily constrain training within the dataset distribution, resulting in overly conservative policies that struggl…

• Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR

  Zijun Min, Bingshuai Liu, Ante Wang, Long Zhang, Anxiang Zeng, Haibo Zhang, Jinsong Su · 12. Januar 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising framework for optimizing large language models in reasoning tasks. However, existing RLVR algorithms focus on different granularities, and each has complementary strengths and limitations. Group Relative Policy Optimization (GR…

• IIB-LPO: Latent Policy Optimization via Iterative Information Bottleneck

  Huilin Deng, Hongchen Luo, Yue Zhu, Long Li, Zhuoyue Chen, Xinghao Zhao, Ming Li, Jihai Zhang, Mengchang Wang, Yang Cao, Yu Kang · 12. Januar 2026

  Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Model (LLM) reasoning have been hindered by a persistent challenge: exploration collapse. The semantic homogeneity of random rollouts often traps models in narrow, over-optimized behaviors. While existing met…

• CHDP: Cooperative Hybrid Diffusion Policies for Reinforcement Learning in Parameterized Action Space

  Bingyi Liu, Jinbo He, Haiyong Shi, Enshu Wang, Weizhen Han, Jingxiang Hao, Peixi Wang, Zhuangzhuang Zhang · 12. Januar 2026

  Hybrid action space, which combines discrete choices and continuous parameters, is prevalent in domains such as robot control and game AI. However, efficiently modeling and optimizing hybrid discrete-continuous action space remains a fundamental challenge, mainly due to limited policy expressiveness…

• Interactive Distillation for Cooperative Multi-Agent Reinforcement Learning

  Minwoo Cho, Batuhan Altundas, Matthew Gombolay · 12. Januar 2026

  Knowledge distillation (KD) has the potential to accelerate MARL by employing a centralized teacher for decentralized students but faces key bottlenecks. Specifically, there are (1) challenges in synthesizing high-performing teaching policies in complex domains, (2) difficulties when teachers must r…

• Nightmare Dreamer: Dreaming About Unsafe States And Planning Ahead

  Oluwatosin Oseni, Shengjie Wang, Jun Zhu, Micah Corah · 9. Januar 2026

  Reinforcement Learning (RL) has shown remarkable success in real-world applications, particularly in robotics control. However, RL adoption remains limited due to insufficient safety guarantees. We introduce Nightmare Dreamer, a model-based Safe RL algorithm that addresses safety concerns by leverag…

• Enhanced-FQL($\lambda$), an Efficient and Interpretable RL with novel Fuzzy Eligibility Traces and Segmented Experience Replay

  Mohsen Jalaeian-Farimani · 9. Januar 2026

  This paper introduces a fuzzy reinforcement learning framework, Enhanced-FQL($\lambda$), that integrates novel Fuzzified Eligibility Traces (FET) and Segmented Experience Replay (SER) into fuzzy Q-learning with Fuzzified Bellman Equation (FBE) for continuous control tasks. The proposed approach empl…

• Survival Dynamics of Neural and Programmatic Policies in Evolutionary Reinforcement Learning

  Anton Roupassov-Ruiz, Yiyang Zuo · 9. Januar 2026

  In evolutionary reinforcement learning tasks (ERL), agent policies are often encoded as small artificial neural networks (NERL). Such representations lack explicit modular structure, limiting behavioral interpretation. We investigate whether programmatic policies (PERL), implemented as soft, differe…

• Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art

  Timofey Tomashevskiy · 9. Januar 2026

  This work provides a state-of-the-art survey of continual safe online reinforcement learning (COSRL) methods. We discuss theoretical aspects, challenges, and open questions in building continual online safe reinforcement learning algorithms. We provide the taxonomy and the details of continual onlin…