Reinforcement Learning in Robotics

Últimos artículos

• First-order Sobolev Reinforcement Learning

  Fabian Schramm, Nicolas Perrin-Gilbert, Justin Carpentier · 25 de noviembre de 2025

  We propose a refinement of temporal-difference learning that enforces first-order Bellman consistency: the learned value function is trained to match not only the Bellman targets in value but also their derivatives with respect to states and actions. By differentiating the Bellman backup through dif…

• Beneficial Reasoning Behaviors in Agentic Search and Effective Post-training to Obtain Them

  Jiahe Jin, Abhijay Paladugu, Chenyan Xiong · 25 de noviembre de 2025

  Agentic search leverages LLMs to solve complex user information needs by executing a multi-step process of planning, searching, and synthesizing information to provide answers. This paradigm introduces unique challenges for LLMs' agentic reasoning capabilities when interacting with search systems. I…

• Experience-Efficient Model-Free Deep Reinforcement Learning Using Pre-Training

  Ruoxing Yang · 25 de noviembre de 2025

  We introduce PPOPT - Proximal Policy Optimization using Pretraining, a novel, model-free deep-reinforcement-learning algorithm that leverages pretraining to achieve high training efficiency and stability on very small training samples in physics-based environments. Reinforcement learning agents typi…

• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics

  Chenhao Li, Andreas Krause, Marco Hutter · 25 de noviembre de 2025

  Learning robust and generalizable world models is crucial for enabling efficient and scalable robotic control in real-world environments. In this work, we introduce a novel framework for learning world models that accurately capture complex, partially observable, and stochastic dynamics. The propose…

• Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

  Bochen Lyu, Yiyang Jia, Xiaohao Cai, Zhanxing Zhu · 25 de noviembre de 2025

  Transformers can acquire Chain-of-Thought (CoT) capabilities to solve complex reasoning tasks through fine-tuning. Reinforcement learning (RL) and supervised fine-tuning (SFT) are two primary approaches to this end, yet their underlying mechanisms and differences remain theoretically unclear. In thi…

• Non-stationary and Varying-discounting Markov Decision Processes for Reinforcement Learning

  Zhizuo Chen, Theodore T. Allen · 25 de noviembre de 2025

  Algorithms developed under stationary Markov Decision Processes (MDPs) often face challenges in non-stationary environments, and infinite-horizon formulations may not directly apply to finite-horizon tasks. To address these limitations, we introduce the Non-stationary and Varying-discounting MDP (NV…

• Learning Primitive Embodied World Models: Towards Scalable Robotic Learning

  Qiao Sun, Liujia Yang, Wei Tang, Wei Huang, Kaixin Xu, Yongchao Chen, Mingyu Liu, Jiange Yang, Haoyi Zhu, Yating Wang, Tong He, Yilun Chen, Xili Dai, Nanyang Ye, Qinying Gu · 25 de noviembre de 2025

  While video-generation-based embodied world models have gained increasing attention, their reliance on large-scale embodied interaction data remains a key bottleneck. The scarcity, difficulty of collection, and high dimensionality of embodied data fundamentally limit the alignment granularity betwee…

• Learning Robust Social Strategies with Large Language Models

  Dereck Piche, Mohammed Muqeeth, Milad Aghajohari, Juan Duque, Michael Noukhovitch, Aaron Courville · 25 de noviembre de 2025

  As agentic AI becomes more widespread, agents with distinct and possibly conflicting goals will interact in complex ways. These multi-agent interactions pose a fundamental challenge, particularly in social dilemmas, where agents' individual incentives can undermine collective welfare. While reinforc…

• Human-Inspired Multi-Level Reinforcement Learning

  Mingkang Wu, Devin White, Vernon Lawhern, Nicholas R. Waytowich, Yongcan Cao · 25 de noviembre de 2025

  Reinforcement learning (RL), a common tool in decision making, learns control policies from various experiences based on the associated cumulative return/rewards without treating them differently. Humans, on the contrary, often learn to distinguish from discrete levels of performance and extract the…

• Multi-Agent Cross-Entropy Method with Monotonic Nonlinear Critic Decomposition

  Yan Wang, Ke Deng, Yongli Ren · 25 de noviembre de 2025

  Cooperative multi-agent reinforcement learning (MARL) commonly adopts centralized training with decentralized execution (CTDE), where centralized critics leverage global information to guide decentralized actors. However, centralized-decentralized mismatch (CDM) arises when the suboptimal behavior o…

• LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems

  Tianyang Duan, Zongyuan Zhang, Zheng Lin, Songxiao Guo, Xiuxian Guan, Guangyu Wu, Zihan Fang, Haotian Meng, Xia Du, Ji-Zhe Zhou, Heming Cui, Jun Luo, Yue Gao · 25 de noviembre de 2025

  Multi-agent reinforcement learning (MARL) has been increasingly adopted in many real-world applications. While MARL enables decentralized deployment on resource-constrained edge devices, it suffers from severe non-stationarity due to the synchronous updates of agent policies. This non stationarity r…

• Distributionally Robust Free Energy Principle for Decision-Making

  Allahkaram Shafiei, Hozefa Jesawada, Karl Friston, Giovanni Russo · 25 de noviembre de 2025

  Despite their groundbreaking performance, autonomous agents can misbehave when training and environmental conditions become inconsistent, with minor mismatches leading to undesirable behaviors or even catastrophic failures. Robustness towards these training-environment ambiguities is a core requirem…

• Leveraging LLMs for reward function design in reinforcement learning control tasks

  Franklin Cardenoso, Wouter Caarls · 25 de noviembre de 2025

  The challenge of designing effective reward functions in reinforcement learning (RL) represents a significant bottleneck, often requiring extensive human expertise and being time-consuming. Previous work and recent advancements in large language models (LLMs) have demonstrated their potential for au…

• VADE: Variance-Aware Dynamic Sampling via Online Sample-Level Difficulty Estimation for Multimodal RL

  Zengjie Hu, Jiantao Qiu, Tianyi Bai, Haojin Yang, Binhang Yuan, Qi Jing, Conghui He, Wentao Zhang · 25 de noviembre de 2025

  Group-based policy optimization methods like GRPO and GSPO have become standard for training multimodal models, leveraging group-wise rollouts and relative advantage estimation. However, they suffer from a critical \emph{gradient vanishing} problem when all responses within a group receive identical…

• MOMA-AC: A preference-driven actor-critic framework for continuous multi-objective multi-agent reinforcement learning

  Adam Callaghan, Karl Mason, Patrick Mannion · 25 de noviembre de 2025

  This paper addresses a critical gap in Multi-Objective Multi-Agent Reinforcement Learning (MOMARL) by introducing the first dedicated inner-loop actor-critic framework for continuous state and action spaces: Multi-Objective Multi-Agent Actor-Critic (MOMA-AC). Building on single-objective, single-age…

• Boosting Reinforcement Learning in 3D Visuospatial Tasks Through Human-Informed Curriculum Design

  Markus D. Solbach, John K. Tsotsos · 25 de noviembre de 2025

  Reinforcement Learning is a mature technology, often suggested as a potential route towards Artificial General Intelligence, with the ambitious goal of replicating the wide range of abilities found in natural and artificial intelligence, including the complexities of human cognition. While RL had sh…

• 1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities

  Kevin Wang, Ishaan Javali, Micha{\l} Bortkiewicz, Tomasz Trzci\'nski, Benjamin Eysenbach · 25 de noviembre de 2025

  Scaling up self-supervised learning has driven breakthroughs in language and vision, yet comparable progress has remained elusive in reinforcement learning (RL). In this paper, we study building blocks for self-supervised RL that unlock substantial improvements in scalability, with network depth ser…

• Convergence and stability of Q-learning in Hierarchical Reinforcement Learning

  Massimiliano Manenti, Andrea Iannelli · 24 de noviembre de 2025

  Hierarchical Reinforcement Learning promises, among other benefits, to efficiently capture and utilize the temporal structure of a decision-making problem and to enhance continual learning capabilities, but theoretical guarantees lag behind practice. In this paper, we propose a Feudal Q-learning sch…

• Meta-World+: An Improved, Standardized, RL Benchmark

  Reginald McLean, Evangelos Chatzaroulas, Luc McCutcheon, Frank R\"oder, Tianhe Yu, Zhanpeng He, K. R. Zentner, Ryan Julian, J K Terry, Isaac Woungang, Nariman Farsad, Pablo Samuel Castro · 24 de noviembre de 2025

  Meta-World is widely used for evaluating multi-task and meta-reinforcement learning agents, which are challenged to master diverse skills simultaneously. Since its introduction however, there have been numerous undocumented changes which inhibit a fair comparison of algorithms. This work strives to …

• Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization

  Vinay Kanakeri, Shivam Bajaj, Ashwin Verma, Vijay Gupta, Aritra Mitra · 24 de noviembre de 2025

  It is known that reinforcement learning (RL) is data-hungry. To improve sample-efficiency of RL, it has been proposed that the learning algorithm utilize data from 'approximately similar' processes. However, since the process models are unknown, identifying which other processes are similar poses a …

• Neighbor GRPO: Contrastive ODE Policy Optimization Aligns Flow Models

  Dailan He, Guanlin Feng, Xingtong Ge, Yazhe Niu, Yi Zhang, Bingqi Ma, Guanglu Song, Yu Liu, Hongsheng Li · 24 de noviembre de 2025

  Group Relative Policy Optimization (GRPO) has shown promise in aligning image and video generative models with human preferences. However, applying it to modern flow matching models is challenging because of its deterministic sampling paradigm. Current methods address this issue by converting Ordina…

• MIR: Efficient Exploration in Episodic Multi-Agent Reinforcement Learning via Mutual Intrinsic Reward

  Kesheng Chen, Wenjian Luo, Bang Zhang, Zeping Yin, Zipeng Ye · 24 de noviembre de 2025

  Episodic rewards present a significant challenge in reinforcement learning. While intrinsic reward methods have demonstrated effectiveness in single-agent rein-forcement learning scenarios, their application to multi-agent reinforcement learn-ing (MARL) remains problematic. The primary difficulties …

• Efficient Solution and Learning of Robust Factored MDPs

  Yannik Schnitzer, Alessandro Abate, David Parker · 21 de noviembre de 2025

  Robust Markov decision processes (r-MDPs) extend MDPs by explicitly modelling epistemic uncertainty about transition dynamics. Learning r-MDPs from interactions with an unknown environment enables the synthesis of robust policies with provable (PAC) guarantees on performance, but this can require a …

• A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms

  Ali Murtaza Caunhye, Asad Jeewa · 21 de noviembre de 2025

  The field of Offline Reinforcement Learning (RL) aims to derive effective policies from pre-collected datasets without active environment interaction. While traditional offline RL algorithms like Conservative Q-Learning (CQL) and Implicit Q-Learning (IQL) have shown promise, they often face challeng…

• Limitations of Scalarisation in MORL: A Comparative Study in Discrete Environments

  Muhammad Sa'ood Shah, Asad Jeewa · 21 de noviembre de 2025

  Scalarisation functions are widely employed in MORL algorithms to enable intelligent decision-making. However, these functions often struggle to approximate the Pareto front accurately, rendering them unideal in complex, uncertain environments. This study examines selected Multi-Objective Reinforcem…