Reinforcement Learning in Robotics

Últimos artículos

• Adaptable Hindsight Experience Replay for Search-Based Learning

  Alexandros Vazaios, Jannis Brugger, Cedric Derstroff, Kristian Kersting, Mira Mezini · 6 de noviembre de 2025

  AlphaZero-like Monte Carlo Tree Search systems, originally introduced for two-player games, dynamically balance exploration and exploitation using neural network guidance. This combination makes them also suitable for classical search problems. However, the original method of training the network wi…

• Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments

  Bryan L. M. de Oliveira, Felipe V. Frujeri, Marcos P. C. M. Queiroz, Luana G. B. Martins, Telma W. de L. Soares, Luckeciano C. Melo · 6 de noviembre de 2025

  Group Relative Policy Optimization (GRPO) has emerged as a scalable alternative to Proximal Policy Optimization (PPO) by eliminating the learned critic and instead estimating advantages through group-relative comparisons of trajectories. This simplification raises fundamental questions about the nec…

• Scaling Multi-Agent Environment Co-Design with Diffusion Models

  Hao Xiang Li, Michael Amir, Amanda Prorok · 6 de noviembre de 2025

  The agent-environment co-design paradigm jointly optimises agent policies and environment configurations in search of improved system performance. With application domains ranging from warehouse logistics to windfarm management, co-design promises to fundamentally change how we deploy multi-agent sy…

• Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL

  Lipeng Zu, Hansong Zhou, Xiaonan Zhang · 6 de noviembre de 2025

  Offline reinforcement learning (RL) enables training from fixed data without online interaction, but policies learned offline often struggle when deployed in dynamic environments due to distributional shift and unreliable value estimates on unseen state-action pairs. We introduce Behavior-Adaptive Q…

• Reinforcement Learning Using known Invariances

  Alexandru Cioba, Aya Kayal, Laura Toni, Sattar Vakili, Alberto Bernacchia · 6 de noviembre de 2025

  In many real-world reinforcement learning (RL) problems, the environment exhibits inherent symmetries that can be exploited to improve learning efficiency. This paper develops a theoretical and algorithmic framework for incorporating known group symmetries into kernel-based RL. We propose a symmetry…

• Shift Before You Learn: Enabling Low-Rank Representations in Reinforcement Learning

  Bastien Dubail, Stefan Stojanovic, Alexandre Prouti\`ere · 6 de noviembre de 2025

  Low-rank structure is a common implicit assumption in many modern reinforcement learning (RL) algorithms. For instance, reward-free and goal-conditioned RL methods often presume that the successor measure admits a low-rank representation. In this work, we challenge this assumption by first remarking…

• Shrinking the Variance: Shrinkage Baselines for Reinforcement Learning with Verifiable Rewards

  Guanning Zeng, Zhaoyi Zhou, Daman Arora, Andrea Zanette · 6 de noviembre de 2025

  Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm for post-training large reasoning models (LRMs) using policy-gradient methods such as GRPO. To stabilize training, these methods typically center trajectory rewards by subtracting the empirical mean for each pro…

• Value of Information-Enhanced Exploration in Bootstrapped DQN

  Stergios Plataniotis, Charilaos Akasiadis, Georgios Chalkiadakis · 6 de noviembre de 2025

  Efficient exploration in deep reinforcement learning remains a fundamental challenge, especially in environments characterized by high-dimensional states and sparse rewards. Traditional exploration strategies that rely on random local policy noise, such as $\epsilon$-greedy and Boltzmann exploration…

• Going Beyond Expert Performance via Deep Implicit Imitation Reinforcement Learning

  Iason Chrysomallis, Georgios Chalkiadakis · 6 de noviembre de 2025

  Imitation learning traditionally requires complete state-action demonstrations from optimal or near-optimal experts. These requirements severely limit practical applicability, as many real-world scenarios provide only state observations without corresponding actions and expert performance is often s…

• Periodic Skill Discovery

  Jonghae Park, Daesol Cho, Jusuk Lee, Dongseok Shim, Inkyu Jang, H. Jin Kim · 6 de noviembre de 2025

  Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependence between states and skills or maximizing…

• Towards Formalizing Reinforcement Learning Theory

  Shangtong Zhang · 6 de noviembre de 2025

  In this paper, we formalize the almost sure convergence of $Q$-learning and linear temporal difference (TD) learning with Markovian samples using the Lean 4 theorem prover based on the Mathlib library. $Q$-learning and linear TD are among the earliest and most influential reinforcement learning (RL)…

• Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances

  Iason Chrysomallis, Georgios Chalkiadakis · 6 de noviembre de 2025

  Imitation learning (IL) enables agents to acquire skills by observing and replicating the behavior of one or multiple experts. In recent years, advances in deep learning have significantly expanded the capabilities and scalability of imitation learning across a range of domains, where expert data ca…

• Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning

  Hongjoon Ahn, Heewoong Choi, Jisu Han, Taesup Moon · 5 de noviembre de 2025

  Offline goal-conditioned reinforcement learning (GCRL) offers a practical learning paradigm in which goal-reaching policies are trained from abundant state-action trajectory datasets without additional environment interaction. However, offline GCRL still struggles with long-horizon tasks, even with …

• Learning Interactive World Model for Object-Centric Reinforcement Learning

  Fan Feng, Phillip Lippe, Sara Magliacane · 5 de noviembre de 2025

  Agents that understand objects and their interactions can learn policies that are more robust and transferable. However, most object-centric RL methods factor state by individual objects while leaving interactions implicit. We introduce the Factored Interactive Object-Centric World Model (FIOC-WM), …

• Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning

  Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji · 5 de noviembre de 2025

  Offline reinforcement learning (RL) suffers from extrapolation errors induced by out-of-distribution (OOD) actions. To address this, offline RL algorithms typically impose constraints on action selection, which can be systematically categorized into density, support, and sample constraints. However,…

• Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learning

  Nicolas Castanet, Olivier Sigaud, Sylvain Lamprier · 5 de noviembre de 2025

  Goal-Conditioned Reinforcement Learning (GCRL) enables agents to autonomously acquire diverse behaviors, but faces major challenges in visual environments due to high-dimensional, semantically sparse observations. In the online setting, where agents learn representations while exploring, the latent …

• Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs

  Georgios Tzannetos, Parameswaran Kamalaruban, Adish Singla · 5 de noviembre de 2025

  Training agents to operate under strict constraints during deployment, such as limited resource budgets or stringent safety requirements, presents significant challenges, especially when these constraints render the task complex. In this work, we propose a curriculum learning strategy that gradually…

• From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos

  Xun Wang, Zhuoran Li, Yanshan Lin, Hai Zhong, Longbo Huang · 5 de noviembre de 2025

  Training a team of agents from scratch in multi-agent reinforcement learning (MARL) is highly inefficient, much like asking beginners to play a symphony together without first practicing solo. Existing methods, such as offline or transferable MARL, can ease this burden, but they still rely on costly…

• Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning

  Renos Zabounidis, Aditya Golatkar, Michael Kleinman, Alessandro Achille, Wei Xia, Stefano Soatto · 5 de noviembre de 2025

  We propose Re-FORC, an adaptive reward prediction method that, given a context, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning a…

• Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning

  Beyazit Yalcinkaya, Marcell Vazquez-Chanlatte, Ameesh Shah, Hanna Krasowski, Sanjit A. Seshia · 5 de noviembre de 2025

  We study the problem of learning multi-task, multi-agent policies for cooperative, temporal objectives, under centralized training, decentralized execution. In this setting, using automata to represent tasks enables the decomposition of complex tasks into simpler sub-tasks that can be assigned to ag…

• LEASE: Offline Preference-based Reinforcement Learning with High Sample Efficiency

  Xiao-Yin Liu, Guotao Li, Xiao-Hu Zhou, Zeng-Guang Hou · 5 de noviembre de 2025

  Offline preference-based reinforcement learning (PbRL) provides an effective way to overcome the challenges of designing reward and the high costs of online interaction. However, since labeling preference needs real-time human feedback, acquiring sufficient preference labels is challenging. To solve…

• Directional-Clamp PPO

  Gilad Karpel, Ruida Zhou, Shoham Sabach, Mohammad Ghavamzadeh · 5 de noviembre de 2025

  Proximal Policy Optimization (PPO) is widely regarded as one of the most successful deep reinforcement learning algorithms, known for its robustness and effectiveness across a range of problems. The PPO objective encourages the importance ratio between the current and behavior policies to move to …

• Overcoming Non-stationary Dynamics with Evidential Proximal Policy Optimization

  Abdullah Akg\"ul, Gulcin Baykal, Manuel Hau{\ss}mann, Melih Kandemir · 5 de noviembre de 2025

  Continuous control of non-stationary environments is a major challenge for deep reinforcement learning algorithms. The time-dependency of the state transition dynamics aggravates the notorious stability problems of model-free deep actor-critic architectures. We posit that two properties will play a …

• Closing the Intent-to-Behavior Gap via Fulfillment Priority Logic

  Bassel El Mabsout, Abdelrahman Abdelgawad, Renato Mancuso · 5 de noviembre de 2025

  Practitioners designing reinforcement learning policies face a fundamental challenge: translating intended behavioral objectives into representative reward functions. This challenge stems from behavioral intent requiring simultaneous achievement of multiple competing objectives, typically addressed …

• When Is Diversity Rewarded in Cooperative Multi-Agent Learning?

  Michael Amir, Matteo Bettini, Amanda Prorok · 5 de noviembre de 2025

  The success of teams in robotics, nature, and society often depends on the division of labor among diverse specialists; however, a principled explanation for when such diversity surpasses a homogeneous team is still missing. Focusing on multi-agent task allocation problems, we study this question fr…