Reinforcement Learning in Robotics

Neueste Paper

• RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance

  Shiqiang Gong · 29. Juni 2026

  Offline reinforcement learning enables policy learning from fixed datasets without additional environment interaction, making it appealing for safety-critical applications where online exploration is costly or unsafe. Diffusion-based decision-making methods have recently achieved strong performance …

• Training Observable Control Policies to Expose Agent State Through Actions

  Andres Enriquez Fernandez, John J. Bird · 29. Juni 2026

  Physical or operational constraints often impose communications limitations on autonomous agents. Such limitations complicate monitoring or multiagent coordination. Even when strong communications are absent, some information may still be available. The remainder of the relevant agent state may be r…

• Regularized Reward-Punishment Reinforcement Learning

  Jiexin Wang, Eiji Uchibe · 29. Juni 2026

  We propose KL-Coupled Policy Regularization (KCPR), a policy coordination framework for Reward-Punishment Reinforcement Learning (RPRL). Based on KCPR, we derive KL-Coupled Soft Optimality (KCSO) and develop its deep realization, klDMP. Unlike existing RPRL approaches that optimize reward-seeking an…

• ATOD: Annealed Turn-aware On-policy Distillation for Multi-turn Autonomous Agents

  Qitai Tan, Zefang Zong, Yang Li, Peng Chen · 29. Juni 2026

  Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement. On-policy distillation (OPD) provides dense teacher guidance and typically improves rapidly in the early stage, but its gains saturate once the student approaches the t…

• Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience

  Raymond Yu, William Huey, Mustafa Mukadam, Anusha Nagabandi, Abhishek Gupta · 29. Juni 2026

  Robots trained on real world data tend to be imprecise, slow, and brittle to perturbations. Improving these policies with reinforcement learning (RL) is an appealing alternative, but this process often requires expensive training in the real world. Performing policy improvement in simulation instead…

• Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

  Ilia Larchenko · 26. Juni 2026

  I describe my solution to the LeHome Challenge 2026, an ICRA 2026 competition on bimanual garment folding. The system placed 1st of 62 teams in the online (simulation) round and 2nd in the real-world final. It improves a vision-language-action (VLA) policy with a reinforcement-learning loop. The pol…

• Reinforcement Learning without Ground-Truth Solutions can Improve LLMs

  Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang, Xunpeng Huang, Kun Zhou, Tongtong Liang, Zhewei Yao, Yi-An Ma, Yuxiong He · 26. Juni 2026

  Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution is unknown. We introduce a \textbf{R}anking-\textbf{i}nduced \textbf{VER}ifiable framework (RiVER) t…

• Mesh-RL: Coupled subgrid reinforcement learning

  Behnam Gheshlaghi, Bahador Rashidi, Shahin Atakishiyev · 26. Juni 2026

  Reinforcement learning in large or sparse-reward environments suffers from slow temporal-difference reward propagation, as value information spreads only locally across the state space. We propose Mesh-RL, a spatial domain-decomposition framework inspired by the finite element method and domain deco…

• Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data

  Erhan Bayraktar, Martin Hernandez, Qinxin Yan, Yuhua Zhu · 26. Juni 2026

  This paper addresses model-free continuous-time mean-field control in a setting where the population dynamics evolve continuously according to an unknown McKean-Vlasov stochastic differential equation, while only discrete-time transition data are available. In the model-based formulation, policy eva…

• RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning

  Rongjian Chen, Jianmin Hu, Kejiang Ye, Minxian Xu · 26. Juni 2026

  Large language model (LLM) post-training for reasoning increasingly relies on reinforcement learning with verifiable rewards (RLVR), where models learn from ground-truth feedback on mathematical, logical, and scientific tasks. To enable flexible resource allocation and support heterogeneous training…

• Revisiting Action Factorization for Complex Action Spaces

  Timothy Flavin, Sandip Sen · 26. Juni 2026

  Many real-world control problems involve hybrid discrete-continuous action spaces. For example, steering and signaling in autonomous driving, and aiming and firing in robotics or video-games. Despite real-world hybrid factorization and reinforcement learning framework support for complex action spac…

• Heavy-Ball Q-Learning with Residual Weighting Correction

  Donghwan Lee · 26. Juni 2026

  This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard Q-learning. The same construction is then extended to Q-lear…

• Adaptive Utility driven Resource Orchestration for Resilient AI (AURORA-AI)

  Rahul Umesh Mhapsekar, Ilias Cherkaoui, Lizy Abraham, Indrakshi Dey · 26. Juni 2026

  Modern AI systems are increasingly deployed under non-stationary computational, demographic, and operational conditions in which static resource allocation strategies degrade both predictive performance and human-centric properties such as fairness and explainability. This paper presents AURORA-AI, …

• Automating Potential-based Reward Shaping with Vision Language Model Guidance

  Henrik M\"uller, Daniel Kudenko · 26. Juni 2026

  Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory. Naive reward shaping can induce reward hacking, yielding policies that exploi…

• R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning

  Haobin Qin, Baofeng Zhang, Hidehisa Akiyama, Keisuke Fujii · 26. Juni 2026

  Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform, but its …

• Autoformalization of Agent Instructions into Policy-as-Code

  Adam Mondl, Matthew Maisel, John H. Brock · 26. Juni 2026

  Agent safety in high-stakes domains requires formal policy enforcement, but most existing approaches either rely on probabilistic guardrails (fine-tuned classifiers, prompt-based steering) that offer no formal guarantees, or on hand-coded symbolic enforcement that does not scale to the breadth of re…

• Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning

  Aniruddha Joshi, Niklas Lauffer, Sanjit Seshia · 26. Juni 2026

  Real-world decision-making often requires balancing multiple conflicting objectives, a challenge that standard Reinforcement Learning (RL) frequently addresses by aggregating rewards into a single scalar signal. While effective for simple tasks, this approach often fails to capture the full spectrum…

• IDEA: Insensitive to Dynamics Mismatch via Effect Alignment for Sim-to-Real Transfer in Multi-Agent Control

  Chenlong Liu, Zhuohui Zhang, Xinyan Chen, Zhipeng Wang, Bin Cheng, Bin He · 26. Juni 2026

  Complex multi-agent control tasks remain challenging for traditional rule-based and model-based approaches, motivating the adoption of learning-based methods. However, learning-based methods often struggle with sim-to-real transfer because they rely on accurate dynamics modeling or system identifica…

• Normalizing Flows are Capable Models for Continuous Control

  Raj Ghugare, Benjamin Eysenbach · 26. Juni 2026

  Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models. To this end, RL researchers often choose to pay the price of accommodating these models into their algorithms -- diffus…

• Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It

  Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao · 25. Juni 2026

  Tool use enables large language models (LLMs) to perform complex tasks, and recent agentic reinforcement learning (RL) methods show promise for enhancing model capabilities. However, RL alone often leads to instability or limited gains in tool-use tasks. In our experiments, some models exhibit catas…

• Minimax PAC Bounds for Learning in Exogenous Contextual MDPs

  Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet · 25. Juni 2026

  We study PAC learning in tabular discounted Markov decision processes with exogenous i.i.d. contexts, with discount factor $\gamma$, finite state space $\mathcal X$, action space $\mathcal A$, and context space $\mathcal Z$. At each time step, a context is drawn independently from an unknown distrib…

• Consistent Zero-Shot Imitation with Contrastive Goal Inference

  Kathryn Wantlin, Chongyi Zheng, Benjamin Eysenbach · 25. Juni 2026

  Zero-shot imitation learning requires an agent to reproduce expert behavior from a single demonstration without additional environment interaction or gradient updates at test time. We introduce Contrastive Inverse Reinforcement Learning (CIRL), a self-supervised framework for pre-training zero-shot …

• Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning

  Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe, Eirik M{\o}ller Nilsen, Jim Torresen, Kai Olav Ellefsen, Tobias L{\o}mo · 25. Juni 2026

  When fine-tuning Large Language Models (LLMs), there has been success in minimizing both memory usage and computation with Parameter-Efficient Fine-Tuning (PEFT), like Low Rank Adaptation (LoRA). In this article, we have explored whether this approach is transferable to the world of robotics and Rei…

• The Interplay of Harness Design and Post-Training in LLM Agents

  Kyungmin Kim, Youngbin Choi, Seoyeon Lee, Suhyeon Jun, Dongwoo Kim, Sangdon Park · 25. Juni 2026

  Tool-integrated LLM agents are often wrapped within a harness: the scaffolding that determines which tools are exposed, how they are described, and what auxiliary information accompanies each per-step observation. While agents are routinely post-trained, this scaffolding is typically treated as a fi…

• Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods

  Zhengzhu Liu, Zeming Gao, Haoyuan Qin, Jiawei Hu, Junhao Wu, Miao Zhu, Haipeng Zhang, Chennan Ma, Siqi Shen, Cheng Wang · 25. Juni 2026

  Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances. We trace this issue to stagnant neurons, units whose gradient updates become negligibly small relative to their weights, the…