Reinforcement Learning in Robotics

Latest papers

• Fisher Decorator: Refining Flow Policy via a Local Transport Map

  Xiaoyuan Cheng, Haoyu Wang, Wenxuan Yuan, Ziyan Wang, Zonghao Chen, Li Zeng, Zhuo Sun · 6 May 2026

  Recent advances in flow-based offline reinforcement learning (RL) have achieved strong performance by parameterizing policies via flow matching. However, they still face critical trade-offs among expressiveness, optimality, and efficiency. In particular, existing flow policies interpret the $L_2$ re…

• HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness

  Jianing Wang, Linsen Guo, Zhengyu Chen, Qi Guo, Hongyu Zang, Wenjie Shi, Haoxiang Ma, Xiangyu Xi, Xiaoyu Li, Wei Wang, Xunliang Cai · 6 May 2026

  Recent advances in agentic harness with orchestration frameworks that coordinate multiple agents with memory, skills, and tool use have achieved remarkable success in complex reasoning tasks. However, the underlying mechanism that truly drives performance remains obscured behind intricate system des…

• Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL

  Sudong Wang, Weiquan Huang, Xiaomin Yu, Zuhao Yang, Hehai Lin, Keming Wu, Chaojun Xiao, Chen Chen, Wenxuan Wang, Beier Zhu, Yunjian Zhang, Chengwei Qin · 6 May 2026

  The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR). However, SFT introduces distributional drift that neither preserves the model's original capabilities…

• Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines

  Wilka Carvalho, Sam Hall-McMaster, Honglak Lee, Samuel J. Gershman · 6 May 2026

  Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as Multitask Prepl…

• Optimal Posterior Sampling for Policy Identification in Tabular Markov Decision Processes

  Cyrille Kone, Kevin Jamieson · 6 May 2026

  We study the $(\varepsilon, \delta)$-PAC policy identification problem in finite-horizon episodic Markov Decision Processes. Existing approaches provide finite-time guarantees for approximate settings ($\varepsilon>0$) but suffer from high computational cost, rendering them hard to implement, and al…

• Quality-Aware Exploration Budget Allocation for Cooperative Multi-Agent Reinforcement Learning

  Dahyun Oh, Minhyuk Yoon, H. Jin Kim · 6 May 2026

  Cooperative multi-agent reinforcement learning (MARL) requires agents to discover joint strategies in a combinatorially large state-action space, yet effective coordination configurations are exceedingly rare. Intrinsic motivation, which augments task rewards with novelty bonuses, is a popular appro…

• T$^2$PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning

  Haixin Wang, Hejie Cui, Chenwei Zhang, Xin Liu, Shuowei Jin, Shijie Geng, Xinyang Zhang, Nasser Zalmout, Zhenyu Shi, Yizhou Sun · 6 May 2026

  Recent progress in multi-turn reinforcement learning (RL) has significantly improved reasoning LLMs' performances on complex interactive tasks. Despite advances in stabilization techniques such as fine-grained credit assignment and trajectory filtering, instability remains pervasive and often leads …

• Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learning

  Rufeng Chen, Zhaofan Zhang, Zhejiang Yang, Hechang Chen, Sihong Xie · 6 May 2026

  Offline safe reinforcement learning often requires policies to adapt at deployment time to safety budgets that vary across episodes or change within a single episode. While diffusion-based planners enable flexible trajectory generation, existing guidance schemes often treat reward improvement and co…

• Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation

  Jingchu Gai, Laixi Shi · 6 May 2026

  Multi-agent reinforcement learning (MARL) holds great potential but faces robustness challenges due to environmental uncertainty. To address this, distributionally robust Markov games (RMGs) optimize worst-case performance when the environment deviates from the nominal model within a uncertainty set…

• Discovering Reinforcement Learning Interfaces with Large Language Models

  Akshat Singh Jaswal, Ashish Baghel, Paras Chopra · 6 May 2026

  Reinforcement learning systems rely on environment interfaces that specify observations and reward functions, yet constructing these interfaces for new tasks often requires substantial manual effort. While recent work has automated reward design using large language models (LLMs), these approaches a…

• Parameter-Efficient Distributional RL via Normalizing Flows and a Geometry-Aware Cram\'er Surrogate

  Simo Alami C., Rim Kaddah, Jesse Read, Marie-Paule Cani · 6 May 2026

  Distributional Reinforcement Learning (DistRL) improves upon expectation-based methods by modeling full return distributions, but standard approaches often remain far from parsimonious. Categorical methods (e.g., C51) rely on fixed supports where parameter counts scale linearly with resolution, whil…

• Population-Aware Imitation Learning in Mean-field Games with Common Noise

  Gr\'egoire Lambrecht, Mathieu Lauri\`ere · 6 May 2026

  Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stoch…

• DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment

  Hongbo Jin, Rongpeng Zhu, Zhongjing Du, Xu Jiang, Jingqi Tian, Qiaoman Zhang, Jiayu Ding · 6 May 2026

  Reinforcement learning is crucial for aligning large language models to perform complex reasoning tasks. However, current algorithms such as Group Relative Policy Optimization suffer from coarse grained, sequence level credit assignment, which severely struggles to isolate pivotal reasoning steps wi…

• OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

  Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Da, Paarth Shah, Max Simchowitz · 6 May 2026

  Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning. This work introduces Off-policy Generative Policy Optimization (OGPO), a sample-efficient algorithm for finetuning GCPs that maintains off-policy cr…

• CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making

  Guowei Zou, Haitao Wang, Beiwen Zhang, Boning Zhang, Hejun Wu · 6 May 2026

  Generative models have emerged as a major paradigm for offline multi-agent reinforcement learning (MARL), but existing approaches require many iterative sampling steps. Recent few-step accelerations either distill a joint teacher into independent students or apply averaged velocities independently p…

• Moira: Language-driven Hierarchical Reinforcement Learning for Pair Trading

  Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang, Xueqing Peng, Jimin Huang, Guojun Xiong, Sophia Ananiadou · 6 May 2026

  Many sequential decision-making problems exhibit hierarchical structure, where high-level semantic choices constrain downstream actions and feedback is delayed and ambiguous. Learning in such settings is challenging due to credit assignment: performance degradation may arise from flawed abstractions…

• MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learning

  Haohan Yu, Jinmiao Cong, Shengzhi Wang, Lu Wang, Chanjuan Liu · 5 May 2026

  A key challenge in multi-agent reinforcement learning (MARL) lies in designing learning signals that effectively promote coordination among agents. Designing such signals necessitates the ability to quantify the true, long-term causal influence between agents. To address this, we introduce Multi-ste…

• Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms

  Tian Xu, Zhilong Zhang, Zexuan Chen, Ruishuo Chen, Yihao Sun, Yang Yu · 5 May 2026

  Adversarial imitation learning (AIL), a prominent approach in imitation learning, has achieved significant practical success powered by neural network approximation. However, existing theoretical analyses of AIL are primarily confined to simplified settings, such as tabular and linear function appro…

• Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis

  Tian Xu, Ziniu Li, Yang Yu, Zhi-Quan Luo · 5 May 2026

  Imitation learning learns a policy from expert trajectories. While the expert data is believed to be crucial for imitation quality, it was found that a kind of imitation learning approach, adversarial imitation learning (AIL), can have exceptional performance. With as little as only one expert traje…

• Zero-Shot, Safe and Time-Efficient UAV Navigation via Potential-Based Reward Shaping, Control Lyapunov and Barrier Functions

  Ashik Abrar Naeem, Mohammad Ariful Haque · 5 May 2026

  Autonomous navigation and obstacle avoidance remain a core challenge of modern Unmanned Aerial Vehicles (UAVs). While traditional control methods struggle with the complexity and variability of the environment, reinforcement learning (RL) enables UAVs to learn adaptive behaviors through interaction …

• PACE: Parameter Change for Unsupervised Environment Design

  Fang Yuan, Quanjun Yin, Siqi Shen, Yuxiang Xie, Junqiang Yang, Long Qin, Junjie Zeng, Qinglun Li · 5 May 2026

  Unsupervised Environment Design (UED) offers a promising paradigm for improving reinforcement learning generalization by adaptively shaping training environments, but it requires reliable environment evaluation to remain effective. However, existing UED methods evaluate environments using indirect p…

• Feedback-Normalized Developer Memory for Reinforcement-Learning Coding Agents: A Safety-Gated MCP Architecture

  Mehmet Iscan · 5 May 2026

  Large language model (LLM) coding agents increasingly operate over repositories, terminals, tests, and execution traces across long software-engineering episodes. Persistent memory is useful, but static vector stores or generic retrieval-augmented generation (RAG) are insufficient for reinforcement-…

• Beyond Specialization: Robust Reinforcement Learning Navigation via Procedural Map Generators

  Christian Jestel, Nicolas Bach, Marvin Wiedemann, Jan Finke, Peter Detzner · 5 May 2026

  Deep reinforcement learning (DRL) navigation policies often overfit to the structure of their training environments, as environmental diversity is typically constrained by the manual effort required to design diverse scenarios. While procedural map generation offers scalable diversity, no prior work…

• AdamO: A Collapse-Suppressed Optimizer for Offline RL

  Nan Qiao, Sheng Yue, Shuning Wang, Ju Ren · 5 May 2026

  Offline reinforcement learning (RL) can fail spectacularly when bootstrapped temporal-difference (TD) updates amplify their own errors, driving the critic toward extreme and unusable Q-values. A key counterintuitive insight of this work is that collapse is not only a property of the backup rule or n…

• Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamics

  Maksim Bobrin, Ilya Zisman, Alexander Nikulin, Vladislav Kurenkov, Dmitry Dylov · 5 May 2026

  Behavioral Foundation Models (BFMs) proved successful in producing policies for arbitrary tasks in a zero-shot manner, requiring no test-time training or task-specific fine-tuning. Among the most promising BFMs are the ones that estimate the successor measure learned in an unsupervised way from task…