Reinforcement Learning in Robotics

Neueste Paper

• From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training

  Zishang Jiang, Tingyun Li, Jinyi Han, Xinyi Wang, Sihang Jiang, Yizhou Ying, Xiaojun Meng, Jiansheng Wei, Jiaqing Liang, Yanghua Xiao · 21. Juli 2026

  Reinforcement learning (RL) has become a widely adopted technique for improving large language models (LLMs) on complex tasks. Despite this progress, existing RL methods still face challenges in training agents with longer-horizon interactions. One major bottleneck is distinguishing the contribution…

• Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach

  Siddharth Chandak · 21. Juli 2026

  We establish mean-square and concentration bounds for stochastic approximation (SA) with arbitrary norm contractive mappings, under a multiplicative noise model where the noise may scale affinely with the norm of the iterates, and the iterates are potentially unbounded. These settings arise in reinf…

• WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning

  Ryan Xu, Atlas Zhao, David Bao, Frank Du · 21. Juli 2026

  Long-horizon rollout generation has become the dominant systems bottleneck in agentic reinforcement learning (RL). As agents interact with environments over many turns, trajectories rapidly grow to tens of thousands of tokens, making synchronous RL training increasingly constrained by rollout. We pr…

• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL

  Darshan Deshpande · 21. Juli 2026

  Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments. Hand-curated environments with fixed task and reward difficulties become ineffective signals as model performance improves, and sparse rewards over long horizons induce mode collapse on s…

• Generalised Bellman recurrence and three dualities in sequential decision-making

  Fernando E. Rosas, David Hyland, Daniel Polani · 21. Juli 2026

  What gives the Bellman equation its form? We show that the recursive properties of optimal value functions follow from three conditions: that the dynamics decomposes through sufficient statistics, that the return decomposes recursively, and that the aggregation of uncertainty is compatible with both…

• Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss

  Kemal Devrim Kafadar, Eren \"Ozaltun, Mahmud Efnan \c{S}anl{\i}, Feyza Orak, Emirhan Gazi, Kubilay Ka\u{g}an K\"om\"urc\"u, Naz{\i}m Kemal \"Ure · 21. Juli 2026

  Robust multi-agent coordination relies heavily on inter-agent communication, which is frequently disrupted by physical and environmental constraints in real-world deployments. To maintain operation during these intermittent communication failures, agents can employ internal prediction models to esti…

• Scalable Causal Imitation Learning

  Eylam Tagor, Mingxuan Li, Elias Bareinboim · 21. Juli 2026

  Imitation learning enables learning a policy in an unknown environment with a latent reward signal using expert demonstrations, but it struggles when the imitator's and expert's observations are mismatched and unobserved confounders are present in expert demonstrations. By identifying appropriate ad…

• CIGPO: Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading LLM Agents

  Hao Dou · 21. Juli 2026

  Training multi-turn evidence-reading agents with outcome-only reinforcement learning is unstable because intermediate turns receive little direct credit. In HotpotQA experiments with Qwen2.5-3B-Instruct, GRPO initially improves (standard F1 0.430) but subsequently collapses to 100% format-violating …

• Theoretical Foundations of $\max$@$k$ Reinforcement Learning

  Riccardo Poiani, Martino Bernasconi, Andrea Celli · 21. Juli 2026

  Reinforcement Learning is a cornerstone technique for modern large reasoning models. Usually, for difficult tasks such as code generation and theorem proving, the agent is evaluated by generating $K$ responses rather than sampling a single response, and performance is then measured using a retry-awa…

• Rationalizing Boltzmann Rationality: An Axiomatic Characterization of Entropy-Regularized Policies

  Silviu Pitis · 21. Juli 2026

  The softmax policy $\pi(a \mid s) \propto \exp(\beta Q(s,a))$ is the default model of stochastic choice in reinforcement learning (RL). Various justifications based on robustness, exploration, and optimization have been offered in the RL literature, but none uniquely derives the softmax form from fi…

• Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimization

  Zijian Zhao, Sen Li · 21. Juli 2026

  Multi-agent policy optimization, exemplified by PPO-based methods, is a key branch of cooperative Multi-Agent Reinforcement Learning (MARL). A central design question is how many neighboring agents\footnote{In this paper, "neighbors" refer not only to physical proximity but also to agents whose acti…

• Enhancing Rubric-based RL via Self-Distillation

  Mingxuan Xia, Yuhang Yang, Chao Ye, Shuai Zhu, Shenzhi Yang, Guangcheng Zhu, Yuhang Zhang, Cheng Peng, Haobo Wang, Siqing Wang · 21. Juli 2026

  Rubric-based RL has recently shown promise in improving LLMs on open-ended tasks. A widely recognized limitation of rubric-based RL is limited exploration: criteria that no rollout manages to satisfy (Unexplored Criteria, UC) receive no optimization signal. Recent methods address this by incorporati…

• Distributional Soft Bellman Operator under the Cram\'er Geometry

  Keru Wang, Yixin Deng, Yao Lyu, Stephen Redmond, Shengbo Eben Li · 21. Juli 2026

  Distributional soft policy iteration (DSPI) provides an important framework for combining distributional reinforcement learning (DRL) with maximum-entropy control, in which the policy evaluation step is governed by a distributional soft Bellman operator acting on entropy-regularised returns. Theoret…

• Information-Based Exploration via Random Features for Reinforcement Learning

  Waris Radji, Odalric-Ambrym Maillard · 21. Juli 2026

  Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish. We introduce Random Feature Information Gain (RFIG), grounded in Bayesian kernel methods th…

• Feedback Attribution and Representation Geometry: Metrics for Comparing Individual and Shared Rewards in MARL

  Tasha Pais, Richard Higgins · 21. Juli 2026

  Cooperative multi-agent RL systems routinely use team-averaged rewards, a feedback-attribution choice that gives each agent the team outcome regardless of its individual contribution. We ask whether this leaves a measurable signature, geometric or behavioral, on learned representations. We propose E…

• AGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

  Ruiyi Ding, Jie Li, He Kang, Ziyan Liu, Chengru Song, Yuan chen · 21. Juli 2026

  Group Relative Policy Optimization (GRPO) is a powerful reinforcement learning algorithm for aligning generative models with human preferences. While successful in large language models~\cite{shao2024deepseekmathpushinglimitsmathematical}, its extension to diffusion and flow matching models introduc…

• When to Plan: Learning to Select Between Reactive Control and Deliberative Planning

  Adam Labiosa, Josiah P. Hanna · 21. Juli 2026

  It has long been recognized that humans have the ability to switch between fast, reactive decision-making and slower, deliberative planning. In this paper, we study the question of how to learn this ability, known as meta-reasoning, in artificial agents. We model reactive decision-making as a policy…

• Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?

  Fan Yang, Rui Meng, Yuxin Wen · 21. Juli 2026

  On-policy self-distillation (OPSD) offers a promising approach for training large language models without relying on a separate teacher model. However, its effectiveness on complex agentic tasks remains largely unexplored. In this work, we instantiate Feedback-Augmented Self-Distillation (FA-SD), a …

• Rethinking the Suitability of Reinforcement Learning Algorithms Under Practical Transfer Constraints

  Hany Hamed, Abhishek Naik, Colin Bellinger, A. Rupam Mahmood · 21. Juli 2026

  Transfer-oriented reinforcement learning requires evaluating algorithms along dimensions that go beyond standard sample efficiency. We focus on two dimensions: practical efficiency, which asks whether conclusions about algorithm suitability change under wall-clock rather than interaction-based budge…

• Comprehend, Divide, and Conquer: Feature Subspace Exploration via Multi-Agent Hierarchical Reinforcement Learning

  Weiliang Zhang, Xiaohan Huang, Yi Du, Ziyue Qiao, Qingqing Long, Zhen Meng, Yuanchun Zhou, Meng Xiao · 21. Juli 2026

  Feature selection aims to preprocess the target dataset, find an optimal and most streamlined feature subset, and enhance the downstream machine learning task. Among filter, wrapper, and embedded-based approaches, the reinforcement learning (RL)-based subspace exploration strategy provides a novel o…

• Reinforcement Learning: From Algorithms To Foundation Models

  Zihan Ding · 21. Juli 2026

  Reinforcement learning (RL) provides a framework for sequential decision making under explicit objectives. In its classical form, RL studies how an agent should act to maximise long-term reward in a dynamic environment. In richer settings, the problem extends beyond a single agent and fixed environm…

• PAMD: Structured Adaptive Distances for Bisimulation Representations in Visual Reinforcement Learning

  Daegyeong Roh, Juho Bae, Han-Lim Choi · 21. Juli 2026

  Many visual reinforcement learning (RL) algorithms learn representations by matching latent distances to a behavioral distance induced by reward and transition similarity. In practice, the choice of the latent distance can strongly affect performance: using a fixed, pre-specified global norms (e.g.,…

• Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning

  Ziyi Liu, Grace Zhang · 21. Juli 2026

  Inverse reinforcement learning (IRL) provides a powerful framework for learning from demonstrations. However, real-world tasks often exhibit substantial natural variations (e.g., picking up mugs with varying shapes), making it impractical to collect demonstrations that fully specify a new task under…

• Process Reward Informed Tree Rollout for Effective Multi-Turn RL

  Xintong Li, Sha Li, Yuwei Zhang, Changlong Yu, Rongmei Lin, Hongye Jin, Shuyi Guan, Xin Liu, Linwei Li, Qingyu Yin, Jingbo Shang · 20. Juli 2026

  Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation. In long-horizon agentic tasks, such a uniform rollout strategy can waste budget on uninformative…

• DADiff: Diffusion-Driven Cross-Domain Policy Adaptation for Reinforcement Learning

  Hanyang Chen, Anirudh Satheesh, Longchao Da, Hua Wei · 20. Juli 2026

  Transferring policies across domains poses a vital challenge in reinforcement learning, due to the dynamics mismatch between the source and target domains. In this paper, we consider the setting of online dynamics adaptation, where policies are trained in the source domain with sufficient data, whil…