Reinforcement Learning in Robotics

Neueste Paper

• Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomy

  Aditya Gahlawat, Ahmed Aboudonia, Sandeep Banik, Naira Hovakimyan, Nikolai Matni, Aaron D. Ames, Gioele Zardini, Alberto Speranzon · 22. Dezember 2025

  Imitation learning (IL) enables autonomous behavior by learning from expert demonstrations. While more sample-efficient than comparative alternatives like reinforcement learning, IL is sensitive to compounding errors induced by distribution shifts. There are two significant sources of distribution s…

• A Theoretical Analysis of State Similarity Between Markov Decision Processes

  Zhenyu Tao, Wei Xu, Xiaohu You · 22. Dezember 2025

  The bisimulation metric (BSM) is a powerful tool for analyzing state similarities within a Markov decision process (MDP), revealing that states closer in BSM have more similar optimal value functions. While BSM has been successfully utilized in reinforcement learning (RL) for tasks like state repres…

• Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs

  Junbo Li, Peng Zhou, Rui Meng, Meet P. Vadera, Lihong Li, Yang Li · 22. Dezember 2025

  Reinforcement learning (RL) has re-emerged as a natural approach for training interactive LLM agents in real-world environments. However, directly applying the widely used Group Relative Policy Optimization (GRPO) algorithm to multi-turn tasks exposes notable limitations, particularly in scenarios r…

• Reinforcement Learning for Self-Improving Agent with Skill Library

  Jiongxiao Wang, Qiaojing Yan, Yawei Wang, Yijun Tian, Soumya Smruti Mishra, Zhichao Xu, Megha Gandhi, Panpan Xu, Lin Lee Cheong · 22. Dezember 2025

  Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in complex reasoning and multi-turn interactions but struggle to continuously improve and adapt when deployed in new environments. One promising approach is implementing skill libraries that allow agents to learn, vali…

• Learning to Plan, Planning to Learn: Adaptive Hierarchical RL-MPC for Sample-Efficient Decision Making

  Toshiaki Hori, Jonathan DeCastro, Deepak Gopinath, Avinash Balachandran, Guy Rosman · 22. Dezember 2025

  We propose a new approach for solving planning problems with a hierarchical structure, fusing reinforcement learning and MPC planning. Our formulation tightly and elegantly couples the two planning paradigms. It leverages reinforcement learning actions to inform the MPPI sampler, and adaptively aggr…

• GB-DQN: Gradient Boosted DQN Models for Non-stationary Reinforcement Learning

  Chang-Hwan Lee, Chanseung Lee · 22. Dezember 2025

  Non-stationary environments pose a fundamental challenge for deep reinforcement learning, as changes in dynamics or rewards invalidate learned value functions and cause catastrophic forgetting. We propose \emph{Gradient-Boosted Deep Q-Networks (GB-DQN)}, an adaptive ensemble method that addresses mo…

• Conservative Bias in Multi-Teacher Learning: Why Agents Prefer Low-Reward Advisors

  Maher Mesto, Francisco Cruz · 22. Dezember 2025

  Interactive reinforcement learning (IRL) has shown promise in enabling autonomous agents and robots to learn complex behaviours from human teachers, yet the dynamics of teacher selection remain poorly understood. This paper reveals an unexpected phenomenon in IRL: when given a choice between teacher…

• Translating the Rashomon Effect to Sequential Decision-Making Tasks

  Dennis Gross, J{\o}rn Eirik Betten, Helge Spieker · 22. Dezember 2025

  The Rashomon effect describes the phenomenon where multiple models trained on the same data produce identical predictions while differing in which features they rely on internally. This effect has been studied extensively in classification tasks, but not in sequential decision-making, where an agent…

• DHP: Discrete Hierarchical Planning for Hierarchical Reinforcement Learning Agents

  Shashank Sharma, Janina Hoffmann, Vinay Namboodiri · 22. Dezember 2025

  Hierarchical Reinforcement Learning (HRL) agents often struggle with long-horizon visual planning due to their reliance on error-prone distance metrics. We propose Discrete Hierarchical Planning (DHP), a method that replaces continuous distance estimates with discrete reachability checks to evaluate…

• Planning as Descent: Goal-Conditioned Latent Trajectory Synthesis in Learned Energy Landscapes

  Carlos V\'elez Garc\'ia, Miguel Cazorla, Jorge Pomares · 22. Dezember 2025

  We present Planning as Descent (PaD), a framework for offline goal-conditioned reinforcement learning that grounds trajectory synthesis in verification. Instead of learning a policy or explicit planner, PaD learns a goal-conditioned energy function over entire latent trajectories, assigning low ener…

• Accelerating Multi-modal LLM Gaming Performance via Input Prediction and Mishit Correction

  Ziyang Lin, Zixuan Sun, Sanhorn Chen, Xiaoyang Chen, Roy Zhao · 22. Dezember 2025

  Real-time sequential control agents are often bottlenecked by inference latency. Even modest per-step planning delays can destabilize control and degrade overall performance. We propose a speculation-and-correction framework that adapts the predict-then-verify philosophy of speculative execution to …

• Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning

  Andrew Wagenmaker, Perry Dong, Raymond Tsao, Chelsea Finn, Sergey Levine · 19. Dezember 2025

  Standard practice across domains from robotics to language is to first pretrain a policy on a large-scale demonstration dataset, and then finetune this policy, typically with reinforcement learning (RL), in order to improve performance on deployment domains. This finetuning step has proved critical …

• Enter the Void - Planning to Seek Entropy When Reward is Scarce

  Ashish Sundar, Chunbo Luo, Xiaoyang Wang · 19. Dezember 2025

  Model-based reinforcement learning (MBRL) offers an intuitive way to increase the sample efficiency of model-free RL methods by simultaneously training a world model that learns to predict the future. These models constitute the large majority of training compute and time and they are subsequently u…

• Hypernetworks That Evolve Themselves

  Joachim Winther Pedersen, Erwan Plantec, Eleni Nisioti, Marcello Barylli, Milton Montero, Kathrin Korte, Sebastian Risi · 19. Dezember 2025

  How can neural networks evolve themselves without relying on external optimizers? We propose Self-Referential Graph HyperNetworks, systems where the very machinery of variation and inheritance is embedded within the network. By uniting hypernetworks, stochastic parameter generation, and graph-based …

• INTELLECT-3: Technical Report

  Prime Intellect Team, Mika Senghaas, Fares Obeid, Sami Jaghouar, William Brown, Jack Min Ong, Daniel Auras, Matej Sirovatka, Jannik Straube, Andrew Baker, Sebastian M\"uller, Justus Mattern, Manveer Basra, Aiman Ismail, Dominik Scherm, Cooper Miller, Ameen Patel, Simon Kirsten, Mario Sieg, Christian Reetz, Kemal Erdem, Vincent Weisser, Johannes Hagemann · 19. Dezember 2025

  We present INTELLECT-3, a 106B-parameter Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning on our end-to-end RL infrastructure stack. INTELLECT-3 achieves state of the art performance for its size across math, code, science and reasoning benchmarks, outperforming …

• Meta-RL Induces Exploration in Language Agents

  Yulun Jiang, Liangze Jiang, Damien Teney, Michael Moor, Maria Brbic · 19. Dezember 2025

  Reinforcement learning (RL) has enabled the training of large language model (LLM) agents to interact with the environment and to solve multi-turn long-horizon tasks. However, the RL-trained agents often struggle in tasks that require active exploration and fail to efficiently adapt from trial-and-e…

• EnviSAgE: A Survey of Environment Scaling for Qualitative Agentic Experience Collection

  Yuchen Huang, Sijia Li, Minghao Liu, Wei Liu, Shijue Huang, Zhiyuan Fan, Hou Pong Chan, Yi R. Fung · 19. Dezember 2025

  LLM-based agents can autonomously accomplish complex tasks across various domains. However, to further cultivate capabilities such as adaptive behavior and long-term decision-making, training on static datasets built from human-level knowledge is insufficient. These datasets are costly to construct …

• Non-Asymptotic Global Convergence of PPO-Clip

  Yin Liu, Qiming Dai, Junyu Zhang, Zaiwen Wen · 19. Dezember 2025

  Reinforcement learning (RL) has gained attention for aligning large language models (LLMs) via reinforcement learning from human feedback (RLHF). The actor-only variants of Proximal Policy Optimization (PPO) are widely applied for their efficiency. These algorithms incorporate a clipping mechanism t…

• PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies

  Arhan Jain, Mingtong Zhang, Kanav Arora, William Chen, Marcel Torne, Muhammad Zubair Irshad, Sergey Zakharov, Yue Wang, Sergey Levine, Chelsea Finn, Wei-Chiu Ma, Dhruv Shah, Abhishek Gupta, Karl Pertsch · 19. Dezember 2025

  A significant challenge for robot learning research is our ability to accurately measure and compare the performance of robot policies. Benchmarking in robotics is historically challenging due to the stochasticity, reproducibility, and time-consuming nature of real-world rollouts. This challenge is …

• Failure Modes of Maximum Entropy RLHF

  \"Omer Veysel \c{C}a\u{g}atan, Bar{\i}\c{s} Akg\"un · 19. Dezember 2025

  In this paper, we show that Simple Preference Optimization (SimPO) can be derived as Maximum Entropy Reinforcement Learning, providing a theoretical foundation for this reference-free method. Motivated by SimPO's strong performance in offline preference optimization, we investigate whether Maximum E…

• Adaptation of Agentic AI

  Pengcheng Jiang, Jiacheng Lin, Zhiyi Shi, Zifeng Wang, Luxi He, Yichen Wu, Ming Zhong, Peiyang Song, Qizheng Zhang, Heng Wang, Xueqiang Xu, Hanwen Xu, Pengrui Han, Dylan Zhang, Jiashuo Sun, Chaoqi Yang, Kun Qian, Tian Wang, Changran Hu, Manling Li, Quanzheng Li, Hao Peng, Sheng Wang, Jingbo Shang, Chao Zhang, Jiaxuan You, Liyuan Liu, Pan Lu, Yu Zhang, Heng Ji, Yejin Choi, Dawn Song, Jimeng Sun, Jiawei Han · 19. Dezember 2025

  Cutting-edge agentic AI systems are built on foundation models that can be adapted to plan, reason, and interact with external tools to perform increasingly complex and specialized tasks. As these systems grow in capability and scope, adaptation becomes a central mechanism for improving performance,…

• TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration

  Ye Li, Jiahe Feng, Yuan Meng, Kangye Ji, Chen Tang, Xinwan Wen, Shutao Xia, Zhi Wang, Wenwu Zhu · 19. Dezember 2025

  Diffusion Policy (DP) excels in embodied control but suffers from high inference latency and computational cost due to multiple iterative denoising steps. The temporal complexity of embodied tasks demands a dynamic and adaptable computation mode. Static and lossy acceleration methods, such as quanti…

• Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections

  Niklas Lauffer, Xiang Deng, Srivatsa Kundurthy, Brad Kenstler, Jeff Da · 18. Dezember 2025

  A popular paradigm for training LM agents relies on imitation learning, fine-tuning on expert trajectories. However, we show that the off-policy nature of imitation learning for multi-turn LM agents suffers from the fundamental limitation known as covariate shift: as the student policy's behavior di…

• Spectral Representation-based Reinforcement Learning

  Chenxiao Gao, Haotian Sun, Na Li, Dale Schuurmans, Bo Dai · 18. Dezember 2025

  In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies, value functions, and dynamics models. Although powerful approximations such as neural networks offer great expressivene…

• Automatic Reward Shaping from Multi-Objective Human Heuristics

  Yuqing Xie, Jiayu Chen, Wenhao Tang, Ya Zhang, Chao Yu, Yu Wang · 18. Dezember 2025

  Designing effective reward functions remains a central challenge in reinforcement learning, especially in multi-objective environments. In this work, we propose Multi-Objective Reward Shaping with Exploration (MORSE), a general framework that automatically combines multiple human-designed heuristic …