Reinforcement Learning in Robotics

Neueste Paper

• What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents

  Xiaozhe Li, Tianyi Lyu, Yang Li, Yichuan Ma, Peiji Li, Linyang Li, Qipeng Guo, Dahua Lin, Kai Chen · 20. Mai 2026

  Reinforcement learning can train LLM agents from sparse task rewards, but long-horizon credit assignment remains challenging: a single success-or-failure signal must be distributed across many actions. Existing methods rely on trajectory-level rewards or proxy signals, without fully leveraging per-s…

• Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints

  Jiayu Li, Enpei Zhang, Dawei Zhou, Elynn Chen, Yujun Yan · 20. Mai 2026

  We study workflow learning in a setting where specialized agents hand off control through a shared artifact, each agent observes only a local function of that artifact and its own private state, and no centralized learner accesses joint trajectories -- the operating regime of multi-agent LLM pipelin…

• Sampling-Based Safe Reinforcement Learning

  Luca Vignola, Bruce D. Lee, Manish Prajapat, Manuel Wendl, Melanie Zeilinger, Andreas Krause, Yarden As · 20. Mai 2026

  Safe exploration remains a fundamental challenge in reinforcement learning (RL), limiting the deployment of RL agents in the real world. We propose Sampling-Based Safe Reinforcement Learning (SBSRL), a model-based RL algorithm that maintains safety throughout the learning process by enforcing constr…

• Neural Configuration-Space Barriers for Manipulation Planning and Control

  Kehan Long, Ki Myung Brian Lee, Nikola Raicevic, Niyas Attasseri, Melvin Leok, Nikolay Atanasov · 20. Mai 2026

  Planning and control for high-dimensional robot manipulators in cluttered dynamic environments require computational efficiency and robust safety guarantees. Inspired by recent advances in learning configuration-space distance functions (CDFs) as representations of robot bodies, we propose a unified…

• Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraints

  Timofey Tomashevskiy · 20. Mai 2026

  Safe reinforcement learning in nonstationary environments requires safety mechanisms that adapt as environmental conditions change. Standard safe reinforcement learning methods often assume fixed constraints or stable environmental conditions, which can become inadequate under distribution shift. We…

• Implicit Action Chunking for Smooth Continuous Control

  Bosun Liang, Shuo Pei, Zirui Chen, Chuanzhi Fan, Chen Sun, Yuankai Wu, Huachun Tan, Yong Wang · 20. Mai 2026

  Reinforcement learning often produces high-frequency oscillatory control signals that undermine the safety and stability required for physical deployment. Explicit action chunking addresses this by predicting fixed-horizon trajectories but scales the policy output dimension proportionally with the h…

• Beyond Mode Collapse: Distribution Matching for Diverse Reasoning

  Xiaozhe Li, Yang Li, Xinyu Fang, Shengyuan Ding, Peiji Li, Yongkang Chen, Yichuan Ma, Tianyi Lyu, Linyang Li, Dahua Lin, Qipeng Guo, Qingwen Liu, Kai Chen · 20. Mai 2026

  On-policy reinforcement learning methods like GRPO suffer from mode collapse: they exhibit reduced solution diversity, concentrating probability mass on a single solution once discovered and ceasing exploration of alternative strategies. We show this stems from reverse KL minimization's mode-seeking…

• Composition of Memory Experts for Diffusion World Models

  Sebastian Stapf, Pablo Acuaviva Huertos, Aram Davtyan, Paolo Favaro · 20. Mai 2026

  World models aim to predict plausible futures consistent with past observations, a capability central to planning and decision-making in reinforcement learning. Yet, existing architectures face a fundamental memory trade-off: transformers preserve local detail but are bottlenecked by quadratic atten…

• GAE Falls Short in Imperfect-Information Self-Play Reinforcement Learning

  Zhiyuan Fan, Gabriele Farina · 20. Mai 2026

  Competitive multi-agent reinforcement learning in imperfect-information games requires agents to act under partial observability and against adversarial opponents, necessitating stochastic policies. While self-play reinforcement learning with Proximal Policy Optimization (PPO) has achieved strong em…

• SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learning

  Nikunj Gupta, James Zachary Hare, Jesse Milzman, Rajgopal Kannan, Viktor Prasanna · 20. Mai 2026

  Cooperative multi-agent reinforcement learning agents that act on partial local observations face a fundamental information bottleneck: the knowledge needed to select jointly optimal actions is scattered across the team, yet each agent must commit to a decision without access to its teammates' obser…

• JAXenstein: Accelerated Benchmarking for First-Person Environments

  Ruo Yu Tao, George Konidaris · 20. Mai 2026

  The progression of reinforcement learning algorithms have been driven by challenging benchmarks. The rate in which a researcher can iterate on a problem setting directly impacts the speed of algorithm development. Modern machine learning has produced tools that allow for fast and scalable algorithm …

• CEPO: RLVR Self-Distillation using Contrastive Evidence Policy Optimization

  Ahmed Heakl, Abdelrahman M. Shaker, Youssef Mohamed, Rania Elbadry, Omar Fetouh, Fahad Shahbaz Khan, Salman Khan · 20. Mai 2026

  When a model produces a correct solution under reinforcement learning with verifiable rewards (RLVR), every token receives the same reward signal regardless of whether it was a decisive reasoning step or a grammatical filler. A natural fix is to condition the model on the correct answer as a teacher…

• Learning Efficient Guardrails for Compliance

  Xiaofei Wen, Wenjie Jacky Mo, Yanan Xie, Peng Qi, Muhao Chen · 20. Mai 2026

  Autonomous web agents are increasingly deployed for long-horizon tasks, yet their ability to adhere to real-world policies remains critically underexplored compared to standard safety objectives. To address this gap, we introduce PolicyGuardBench, a benchmark of 60k policy-trajectory pairs designed …

• Metric-Gradient Projection for Stable Multi-Agent Policy Learning

  Zuyuan Zhang, Sizhe Tang, Mahdi Imani, Tian Lan · 20. Mai 2026

  General-sum multi-agent learning is often governed by a stacked update field in which each agent's policy update changes the optimization landscape faced by the others. This coupling can entangle an integrable component of collective improvement with cyclic interaction dynamics, leading to slow or u…

• From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails

  Ravi Pandya, Madison Bland, Duy P. Nguyen, Changliu Liu, Jaime Fern\'andez Fisac, Andrea Bajcsy · 20. Mai 2026

  Generative AI systems are increasingly assisting and acting on behalf of end users in practical settings, from digital shopping assistants to next-generation autonomous cars. In this context, safety is no longer about blocking harmful content, but about preempting downstream hazards like financial o…

• ISEP: Implicit Support Expansion for Offline Reinforcement Learning via Stochastic Policy Optimization

  Yifei Chen, Shaoqin Zhu, Xiaoqiang Ji · 19. Mai 2026

  Offline reinforcement learning methods typically enforce strict constraints to ensure safety; yet this rigidity often prevents the discovery of optimal behaviors outside the immediate support of the behavior policy. To address this, we propose Implicit Support Expansion via stochastic Policy optimiz…

• EfficientTDMPC: Improved MPC Objectives for Sample-Efficient Continuous Control

  Thomas Evers, Cristian Meo, Wendelin Bohmer, Justin Dauwels, Yaniv Oren · 19. Mai 2026

  We introduce EfficientTDMPC, a sample-efficient model-based reinforcement learning method for continuous control built on the TD-MPC family of algorithms. Central to this family is a planner that aims to find an action sequence that maximizes the estimated return. The return is estimated using a lea…

• Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework

  Chengpeng Hu, Yingqian Zhang, Hendrik Baier · 19. Mai 2026

  Deep reinforcement learning (DRL) has recently emerged as a promising approach to solve combinatorial optimization problems such as job shop scheduling. However, the policies learned by DRL are typically represented by deep neural networks (DNNs), whose opaque neural architectures and non-interpreta…

• Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shift

  Surbhi Goel, Jonathan Pei, James Wang · 19. Mai 2026

  Behavior cloning provides strong imitation learning guarantees when training and test environments share the same dynamics. However, in many deployment settings the test environment's transitions differ from training, and classical offline IL offers no recourse: the learner must commit to an action …

• DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models

  Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade, Lokesh Boominathan, Manuel R. Ciosici, Yizhe Zhang, Irina Belousova · 19. Mai 2026

  Diffusion large language models are a compelling alternative to autoregressive models, yet existing RL methods for diffusion treat all denoising steps as equally important and rely on biased, high-variance likelihood estimates. We identify two fundamental weaknesses: the absence of temporal credit a…

• When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisited

  Rishabh Agrawal, Rahul Jain, Ashutosh Nayyar · 19. Mai 2026

  Behavior Foundation Models (BFMs) enable scalable imitation learning (IL) by pretraining task-agnostic representations that can be rapidly adapted to new tasks. However, existing BFMs assume fixed environment dynamics, limiting their robustness under real-world shifts such as changes in friction, ac…

• DiPRL: Learning Discrete Programmatic Policies via Architecture Entropy Regularization

  Chengpeng Hu, Yingqian Zhang, Hendrik Baier · 19. Mai 2026

  Programmatic reinforcement learning (PRL) offers an interpretable alternative to deep reinforcement learning by representing policies as human-readable and -editable programs. While gradient-based methods have been developed to optimize continuous relaxations of programs, they face a significant per…

• Reducing Credit Assignment Variance via Counterfactual Reasoning Paths

  Fei Ding, Yongkang Zhang, Yeling Peng, Youwei Wang, Guoxiong Zhou, Zijian Zeng · 19. Mai 2026

  Reinforcement learning for multi-step reasoning with large language models (LLMs) often relies on sparse terminal rewards, leading to poor credit assignment conditions where the final feedback is evenly propagated across all intermediate decisions. This results in high gradient variance, unstable tr…

• MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings

  Himchan Hwang, Hyeokju Jeong, Gene Chung, Seungyeon Kim, Sangwoong Yoon, Frank Chongwoo Park · 19. Mai 2026

  We propose MATE, a simple yet effective memory architecture for solving Contextual Markov Decision Processes (CMDPs), a family of MDPs parameterized by an unobserved context. In CMDPs, an optimal agent can adapt online by maintaining the posterior belief over contexts. MATE replaces this intractable…

• HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents

  Woongyeng Yeo, Yumin Choi, Taekyung Ki, Sung Ju Hwang · 19. Mai 2026

  Training long-horizon LLM agents with reinforcement learning is challenging because sparse outcome rewards reveal whether a task succeeds, but not which intermediate actions caused the outcome or how they should be corrected. Recent methods alleviate this issue by generating rewards or textual hints…