Reinforcement Learning in Robotics

Neueste Paper

• Balancing Plasticity and Stability with Fast and Slow Successor Features

  Raymond Chua, Doina Precup, Blake Richards · 27. Mai 2026

  A hallmark of intelligence is the ability to adapt in non-stationary environments, yet deep Reinforcement Learning (RL) agents often struggle in such settings. Prior studies introduce non-stationarity through abrupt shifts in features or dynamics, whereas real-world environments often evolve gradual…

• Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning

  Zihong Gao, Hongjian Liang, Lei Hao, Liangjun Ke · 27. Mai 2026

  Communication is essential for coordination in \emph{cooperative} multi-agent reinforcement learning under partial observability, yet \emph{cross-timestep} delays cause messages to arrive multiple timesteps after generation, inducing temporal misalignment and making information stale when consumed. …

• Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning

  Xin Cheng, Shuo He, Lang Feng, HaiYang Xu, Ming Yan, Lei Feng, Bo An · 27. Mai 2026

  Group-based reinforcement learning (RL) methods have achieved remarkable success in improving the performance of large language models (LLMs) and have been rapidly extended to agentic tasks. However, their credit assignment relies heavily on coarse-grained trajectory-level attribution according to f…

• Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generation

  Longwen Wang, Yirui Liu, Xuan'er Wu, Xiaohui Hu, Yuankai Fan, Kaidong Yu, Qizhen Weng, Wei Xi, Xuelong Li · 27. Mai 2026

  Effective reward design is a central challenge in Reinforcement Learning (RL) for code generation. Mainstream test-suite-level outcome rewards enforce functional correctness but induce sparsity, while external Reward Models (RMs) provide dense supervision at the cost of misalignment and additional o…

• Distributed Control of Network Systems in the Space of Stabilizing Graph Neural Network Policies

  John Cao, Luca Furieri · 27. Mai 2026

  We study distributed control of networked systems through reinforcement learning, where neural policies must be simultaneously scalable, expressive and stabilizing. We introduce a policy parameterization that embeds Graph Neural Networks (GNNs) into a Youla-like magnitude-direction parameterization,…

• Continual Model-Based Reinforcement Learning with Hypernetworks

  Yizhou Huang, Kevin Xie, Homanga Bharadhwaj, Florian Shkurti · 27. Mai 2026

  Effective planning in model-based reinforcement learning (MBRL) and model-predictive control (MPC) relies on the accuracy of the learned dynamics model. In many instances of MBRL and MPC, this model is assumed to be stationary and is periodically re-trained from scratch on state transition experienc…

• StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning

  Yanfei Zhang, Xu Lin, Chenglin Wu · 27. Mai 2026

  Reinforcement learning for multi-turn agents suffers from a credit-assignment mismatch: rewards are sparse and trajectory-level, while success often hinges on a few local decisions. Existing online policy distillation (OPD) provides denser token-level supervision, but typically treats heterogeneous …

• Bilevel Optimization over Saddle Points of Zero-Sum Markov Games

  Zihao Zheng, Irwin King, Songtao Lu · 27. Mai 2026

  Reinforcement learning (RL) often has a hierarchical structure, where an upper-level (UL) learner selects model parameters and a lower-level (LL) decision-making process responds, naturally leading to a bilevel optimization problem. Most existing bilevel RL methods assume a single-policy LL Markov d…

• Less is More: Early Stopping Rollout for On-Policy Distillation

  Zhou Ziheng, Jiaqi Li, Huacong Tang, Ying Nian Wu, Demetri Terzopoulos · 27. Mai 2026

  On-policy distillation has recently emerged as a promising alternative to standard sequence-level imitation, training a student by scoring its own rollouts with a teacher model. However, we observe ``Off-policy Teacher Decay'' problem in this paradigm: for the later tokens, with student's earlier tr…

• Yes, Q-learning Helps Offline In-Context RL

  Denis Tarasov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Andrei Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Igor Kiselev, Vladislav Kurenkov · 27. Mai 2026

  Existing offline in-context reinforcement learning (ICRL) methods have predominantly relied on supervised training objectives, which are known to have limitations in offline RL settings. In this study, we explore the integration of RL objectives within an offline ICRL framework. Through experiments …

• BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning

  Shijin Gong, Erhan Xu, Kai Ye, Francesco Quinzan, Giulia Livieri, Chengchun Shi · 27. Mai 2026

  Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existing algorithms face a tradeoff between computational efficiency and sample efficiency in value estimation and policy learning. We introduce BASIS, a critic…

• Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints

  Ting Gao, Stavros Orfanoudakis, Nan Lin, Winnie Daamen, Serge Hoogendoorn, Elvin Isufi · 27. Mai 2026

  Balancing policy expressiveness with the exploration-exploitation trade-off is a core challenge in online Reinforcement Learning (RL). While Stochastic Differential Equation (SDE)-based diffusion policies can represent complex, multimodal action distributions, they suffer from two critical limitatio…

• Adversarial Dual On-Policy Distillation from Expressive Flow-based Teacher

  Zhenglin Wan, Jingxuan Wu, Xingrui Yu, Chubin Zhang, Mingcong Lei, Bo An, Ivor W. Tsang, Yang You · 27. Mai 2026

  Learning from demonstrations in embodied control is often cast as behavioral cloning, and recent diffusion or flow-matching policies improve this paradigm by modeling multi-modal expert actions. Yet these methods remain offline supervised learners: the policy is trained only on expert states and rec…

• Probabilistic Recurrent Intention Switching Model

  Wenyuan Sheng, Hao Zhu, Joschka Boedecker · 27. Mai 2026

  Inverse reinforcement learning (IRL) recovers reward functions from observed behavior, yet traditional methods assume a single stationary reward that cannot capture goal switching within an episode. Recent multi-intention IRL methods address this by segmenting trajectories, but model intention trans…

• Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RL

  Sarthak Dayal, Abhinav Peri, Carl Qi, Claas Voelcker, Alexander Levine, Caleb Chuck, Amy Zhang · 27. Mai 2026

  Hierarchical Reinforcement Learning (HRL) promises to solve long-horizon Reinforcement Learning (RL) tasks more efficiently than non-hierarchical counterparts by discovering and reusing temporally-extended skills. However, obtaining skills that are actually reusable remains an open challenge. Toward…

• Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation

  Yee Hin Chong, Jiaming Wu, Youhui Zhang, Peng Qu · 27. Mai 2026

  Large language models (LLMs) have shown strong empirical gains as self-evolving agents for CUDA kernel generation, driven by feedback-conditioned planning across generations. However, how planning decisions attribute and combine heterogeneous feedback signals remains opaque. Standard end-to-end abla…

• Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

  Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu · 27. Mai 2026

  Model-based reinforcement learning (RL) can be effectively supported at scale through the use of world models. However, in practice, scaling such approaches remains fundamentally limited. A commonly recognized challenge is model bias and error compounding, which degrade long-horizon predictions. Bey…

• GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training

  Yuelin Hu, Zhenbo Yu, Zhengxue Cheng, Wei Liu, Li Song · 27. Mai 2026

  Hybrid post-training usually combines supervised fine-tuning and reinforcement learning, but fixed mixing schedules cannot adapt when the relative noise of the two signals changes over time. We propose GAC, a noise-aware controller that derives an adaptive mixing weight from online estimates of grad…

• Ratio-Variance Regularized Policy Optimization

  Yu Luo, Shuo Han, Yihan Hu, Lei Lv, Huaping Liu, Fuchun Sun, Jianye Hao, Dong Li · 27. Mai 2026

  Standard on-policy reinforcement learning relies on heuristic clipping to enforce trust regions, but this mechanism imposes a severe cost by indiscriminately truncating high-return yet high-divergence updates. We demonstrate that explicitly constraining the policy ratio variance provides a principle…

• Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks

  Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira · 27. Mai 2026

  Reinforcement learning with verifiable rewards (RLVR) has become central to post-training reasoning models, yet a key limitation of existing studies is their narrow view of the reasoning space: difficulty is treated as reasoning depth alone, and reward is concentrated on forward deductive state trac…

• Streaming Reinforcement Learning under Partial Observability with Real-Time Recurrent Learning

  Noah Farr, Aryaman Reddi, Carlo D'Eramo, Jan Peters · 26. Mai 2026

  Streaming reinforcement learning has emerged as an online learning paradigm that conforms to the restrictions of natural learning agents that process data incrementally, i.e. with a batch size of 1 and no replay buffer. While streaming RL has recently been shown to scale with deep function approxima…

• Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learning

  Minjae Kwon, Amir Moeini, Shangtong Zhang, Lu Feng · 26. Mai 2026

  Safe in-context reinforcement learning (ICRL) adapts online from interaction history without test-time parameter updates while controlling episode cost under a safety budget. Under out-of-distribution (OOD) deployment shifts, pretraining-only safe ICRL can give poor reward-safety tradeoffs because t…

• A Contractive Feedback Semantics for Reinforcement Learning

  Zuyuan Zhang · 26. Mai 2026

  Discounted reinforcement learning is usually presented through Bellman equations on closed Markov decision processes. This paper develops a compositional view: a one-step decision process is treated as an open stochastic component, and infinite-horizon policy evaluation is obtained by closing a cont…

• Adaptive Human-AI Coordination via Hierarchical Action Disentanglement

  Adnan Ahmad, Bahareh Nakisa, Mohammad Naim Rastgoo · 26. Mai 2026

  Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners. Existing methods often collapse to a single dominant behavior or learn poorly aligned skills, limiting effective coordination. We propose Intrinsic Action Di…

• Acting on the Unseen: Communication-Free Collaborative Filtering for Decentralized Multi-Robot Task Allocation

  Alexander Apartsin, Yigal Meshulam, Yehudit Aperstein · 26. Mai 2026

  Multi-robot task allocation usually assumes some combination of communication, known task models, or a coordinator. We study the opposite extreme, a regime common in practice but overlooked in theory, which we name Zero-Knowledge MRTA (ZK-MRTA): a robot team with no prior knowledge (no task models, …