Reinforcement Learning in Robotics

Latest papers

• Heterogeneous Multi-Agent Reinforcement Learning with Attention for Cooperative and Scalable Feature Transformation

  Tao Zhe, Huazhen Fang, Kunpeng Liu, Qian Lou, Tamzidul Hoque, Dongjie Wang · 1 December 2025

  Feature transformation enhances downstream task performance by generating informative features through mathematical feature crossing. Despite the advancements in deep learning, feature transformation remains essential for structured data, where deep models often struggle to capture complex feature i…

• Emergent Coordination and Phase Structure in Independent Multi-Agent Reinforcement Learning

  Azusa Yamaguchi · 1 December 2025

  A clearer understanding of when coordination emerges, fluctuates, or collapses in decentralized multi-agent reinforcement learning (MARL) is increasingly sought in order to characterize the dynamics of multi-agent learning systems. We revisit fully independent Q-learning (IQL) as a minimal decentral…

• Interactive Groupwise Comparison for Reinforcement Learning from Human Feedback

  Jan Kompatscher, Danqing Shi, Giovanna Varni, Tino Weinkauf, Antti Oulasvirta · 1 December 2025

  Reinforcement learning from human feedback (RLHF) has emerged as a key enabling technology for aligning AI behaviour with human preferences. The traditional way to collect data in RLHF is via pairwise comparisons: human raters are asked to indicate which one of two samples they prefer. We present an…

• Improving Stochastic Action-Constrained Reinforcement Learning via Truncated Distributions

  Roland Stolz, Michael Eichelbeck, Matthias Althoff · 1 December 2025

  In reinforcement learning (RL), it is often advantageous to consider additional constraints on the action space to ensure safety or action relevance. Existing work on such action-constrained RL faces challenges regarding effective policy updates, computational efficiency, and predictable runtime. Re…

• BiCQL-ML: A Bi-Level Conservative Q-Learning Framework for Maximum Likelihood Inverse Reinforcement Learning

  Junsung Park · 1 December 2025

  Offline inverse reinforcement learning (IRL) aims to recover a reward function that explains expert behavior using only fixed demonstration data, without any additional online interaction. We propose BiCQL-ML, a policy-free offline IRL algorithm that jointly optimizes a reward function and a conserv…

• ASTRO: Adaptive Stitching via Dynamics-Guided Trajectory Rollouts

  Hang Yu, Di Zhang, Qiwei Du, Yanping Zhao, Hai Zhang, Guang Chen, Eduardo E. Veas, Junqiao Zhao · 1 December 2025

  Offline reinforcement learning (RL) enables agents to learn optimal policies from pre-collected datasets. However, datasets containing suboptimal and fragmented trajectories present challenges for reward propagation, resulting in inaccurate value estimation and degraded policy performance. While tra…

• OBLR-PO: A Theoretical Framework for Stable Reinforcement Learning

  Zixun Huang, Jiayi Sheng, Zeyu Zheng · 1 December 2025

  Existing reinforcement learning (RL)-based post-training methods for large language models have advanced rapidly, yet their design has largely been guided by heuristics rather than systematic theoretical principles. This gap limits our understanding of the properties of the gradient estimators and t…

• Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning

  Yang Li, Zhiyuan He, Yuxuan Huang, Zhuhanling Xiao, Chao Yu, Meng Fang, Kun Shao, Jun Wang · 1 December 2025

  Recent Vision-Language Models (VLMs) exhibit strong perceptual reasoning abilities, yet they often struggle to adapt efficiently when encountering novel tasks at test time. In contrast, humans leverage the metacognitive model with memory, enabling continuous strategy refinement through metacognitive…

• Vectorized Online POMDP Planning

  Marcus Hoerger, Muhammad Sudrajat, Hanna Kurniawati · 1 December 2025

  Planning under partial observability is an essential capability of autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for planning under partial observability problems, capturing the stochastic effects of actions and the limited information avai…

• Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs

  Yifan Zhou, Sachin Grover, Mohamed El Mistiri, Kamalesh Kalirathnam, Pratyush Kerhalkar, Swaroop Mishra, Neelesh Kumar, Sanket Gaurav, Oya Aran, Heni Ben Amor · 1 December 2025

  Reinforcement Learning (RL) traditionally relies on scalar reward signals, limiting its ability to leverage the rich semantic knowledge often available in real-world tasks. In contrast, humans learn efficiently by combining numerical feedback with language, prior knowledge, and common sense. We intr…

• Real-Time Procedural Learning From Experience for AI Agents

  Dasheng Bi, Yubin Hu, Mohammed N. Nasir · 1 December 2025

  Learning how to do things from trial and error in real time is a hallmark of biological intelligence, yet most LLM-based agents lack mechanisms to acquire procedural knowledge after deployment. We propose Procedural Recall for Agents with eXperiences Indexed by State (PRAXIS), a lightweight post-tra…

• Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs

  Yong Qi, Gabriel Kyebambo, Siyuan Xie, Wei Shen, Shenghui Wang, Bitao Xie, Bin He, Zhipeng Wang, Shuo Jiang · 27 November 2025

  Safety limitations in service robotics across various industries have raised significant concerns about the need for robust mechanisms ensuring that robots adhere to safe practices, thereby preventing actions that might harm humans or cause property damage. Despite advances, including the integratio…

• Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning

  Sid Bharthulwar, Stone Tao, Hao Su · 27 November 2025

  Massively parallel GPU simulation environments have accelerated reinforcement learning (RL) research by enabling fast data collection for on-policy RL algorithms like Proximal Policy Optimization (PPO). To maximize throughput, it is common to use short rollouts per policy update, increasing the upda…

• On the Origin of Algorithmic Progress in AI

  Hans Gundlach, Alex Fogelson, Jayson Lynch, Ana Trisovic, Jonathan Rosenfeld, Anmol Sandhu, Neil Thompson · 27 November 2025

  Algorithms have been estimated to increase AI training FLOP efficiency by a factor of 22,000 between 2012 and 2023 [Ho et al., 2024]. Running small-scale ablation experiments on key innovations from this time period, we are able to account for less than 10x of these gains. Surveying the broader lite…

• Policy Optimization and Multi-agent Reinforcement Learning for Mean-variance Team Stochastic Games

  Junkai Hu, Li Xia · 27 November 2025

  We study a long-run mean-variance team stochastic game (MV-TSG), where each agent shares a common mean-variance objective for the system and takes actions independently to maximize it. MV-TSG has two main challenges. First, the variance metric is neither additive nor Markovian in a dynamic setting. …

• Predictive Safety Shield for Dyna-Q Reinforcement Learning

  Jin Pin, Krasowski Hanna, Vanneaux Elena · 27 November 2025

  Obtaining safety guarantees for reinforcement learning is a major challenge to achieve applicability for real-world tasks. Safety shields extend standard reinforcement learning and achieve hard safety guarantees. However, existing safety shields commonly use random sampling of safe actions or a fixe…

• Evolutionary Prediction Games

  Eden Saig, Nir Rosenfeld · 27 November 2025

  When a prediction algorithm serves a collection of users, disparities in prediction quality are likely to emerge. If users respond to accurate predictions by increasing engagement, inviting friends, or adopting trends, repeated learning creates a feedback loop that shapes both the model and the popu…

• Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learning

  Shanwei Fan · 27 November 2025

  Large language models (LLMs) offer strong high-level planning capabilities for reinforcement learning (RL) by decomposing tasks into subgoals. However, their practical utility is limited by poor planning-execution alignment, which reflects a critical gap between abstract plans and actionable, enviro…

• Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidance

  Bram Silue, Santiago Amaya-Corredor, Patrick Mannion, Lander Willem, Pieter Libin · 27 November 2025

  Adversarial Inverse Reinforcement Learning (AIRL) has shown promise in addressing the sparse reward problem in reinforcement learning (RL) by inferring dense reward functions from expert demonstrations. However, its performance in highly complex, imperfect-information settings remains largely unexpl…

• Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learning

  Yulei Qin, Xiaoyu Tan, Zhengbao He, Gang Li, Haojia Lin, Zongyi Li, Zihan Xu, Yuchen Shi, Siqi Cai, Renting Rui, Shaofei Cai, Yuzheng Cai, Xuan Zhang, Sheng Ye, Ke Li, Xing Sun · 26 November 2025

  Reinforcement learning (RL) is the dominant paradigm for sharpening strategic tool use capabilities of LLMs on long-horizon, sparsely-rewarded agent tasks, yet it faces a fundamental challenge of exploration-exploitation trade-off. Existing studies stimulate exploration through the lens of policy en…

• Learning Massively Multitask World Models for Continuous Control

  Nicklas Hansen, Hao Su, Xiaolong Wang · 26 November 2025

  General-purpose control demands agents that act across many tasks and embodiments, yet research on reinforcement learning (RL) for continuous control remains dominated by single-task or offline regimes, reinforcing a view that online RL does not scale. Inspired by the foundation model recipe (large-…

• SOMBRL: Scalable and Optimistic Model-Based RL

  Bhavya Sukhija, Lenart Treven, Carmelo Sferrazza, Florian D\"orfler, Pieter Abbeel, Andreas Krause · 26 November 2025

  We address the challenge of efficient exploration in model-based reinforcement learning (MBRL), where the system dynamics are unknown and the RL agent must learn directly from online interactions. We propose Scalable and Optimistic MBRL (SOMBRL), an approach based on the principle of optimism in the…

• Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning

  Charlotte Beylier, Hannah Selder, Arthur Fleig, Simon M. Hofmann, Nico Scherf · 26 November 2025

  The learning process of a reinforcement learning (RL) agent remains poorly understood beyond the mathematical formulation of its learning algorithm. To address this gap, we introduce attention-oriented metrics (ATOMs) to investigate the development of an RL agent's attention during training. In a co…

• Optimization and Regularization Under Arbitrary Objectives

  Jared N. Lakhani, Etienne Pienaar · 26 November 2025

  This study investigates the limitations of applying Markov Chain Monte Carlo (MCMC) methods to arbitrary objective functions, focusing on a two-block MCMC framework which alternates between Metropolis-Hastings and Gibbs sampling. While such approaches are often considered advantageous for enabling d…

• Complex Instruction Following with Diverse Style Policies in Football Games

  Chenglu Sun, Shuo Shen, Haonan Hu, Wei Zhou, Chen Chen · 26 November 2025

  Despite advancements in language-controlled reinforcement learning (LC-RL) for basic domains and straightforward commands (e.g., object manipulation and navigation), effectively extending LC-RL to comprehend and execute high-level or abstract instructions in complex, multi-agent environments, such a…