Reinforcement Learning in Robotics

Latest papers

• Hybrid Open-Ended Tri-Evolution Makes Better Deep Researcher

  Hongming Piao, Chi Liu, Mengzhuo Chen, Yan Shu, Derek Li, Ying Wei, Bryan Dai · 15 June 2026

  Deep research and agent evolution serve as de-facto tasks for AI agents in real-world applications toward artificial general intelligence. The former enables autonomous retrieval and integration of information in open-ended environments to tackle open-ended research tasks, yet it is constrained by t…

• HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

  Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, Jian Luan · 15 June 2026

  AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts. Yet today's harnesses remain largely hand-crafted and static: each new model or task still demands bespoke scaffolding, and th…

• Diffusion Policy Optimization without Drifting Apart

  Haozhe Jiang, Haiwen Feng, Pieter Abbeel, Jiantao Jiao, Angjoo Kanazawa, Nika Haghtalab · 15 June 2026

  RL post-training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy-gradient methods are often unstable and cannot achieve reliable policy improvement. We identify the cause as the double-drift phenomenon: optimizing a variational surrogate can let the EL…

• Causal Object-Centric Models for Planning with Monte Carlo Tree Search

  Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik, Aleksandr Panov · 15 June 2026

  We introduce COMET (Causal Object-centric Model for Efficient Tree search), a model-based reinforcement learning algorithm that performs Monte Carlo Tree Search in a slot-structured latent space. COMET pairs a frozen unsupervised object-centric encoder with a transformer-based world model, in which …

• FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Drift

  Yitao Jiang, Luyang Zhao, Muhao Chen, Devin Balkcom · 15 June 2026

  World models in robot learning predict future states from visual observations and actions, enabling agents to reason about the consequences of their controls. However, many action-conditioned models are evaluated in settings where motion is dominated by immediate control, whereas aquatic surface veh…

• Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning

  Pengxin Wang, Lihao Guo, Yi Xie, Bo Liu, Siyang Cao, Jingdi Chen · 15 June 2026

  Cooperative multi-objective multi-agent reinforcement learning (MOMARL) models team decision making under multiple, potentially conflicting objectives. In this setting, conflicts arise not only across objectives but also across agents with different observations, roles, and contributions. We propose…

• Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models

  Ge Wang, Xinyu Tan, Xiang Li, Man Luo, Chengsi Yao, Shenhao Yan, Jiahao Yang, Fan Feng, Honghao Cai, Xiangyuan Wang, Zhixin Mai, Yiming Zhao, Yatong Han, Zhen Li · 15 June 2026

  Vision-language-action (VLA) models are powerful action generators for robot manipulation, but they are typically executed with fixed inference and replanning schedules. This rigidity ignores the uneven difficulty of robot control: contact-rich or uncertain states may need more computation and fresh…

• Utility-Constrained Policy Optimization

  Mehrdad Moghimi, Bernardo Avila Pires · 15 June 2026

  Constrained MDPs (CMDPs) are a widely adopted framework for incorporating safety into RL agents; however, the framework does not support risk-sensitive constraints. This can be problematic: For example, CMDPs allow for optimal solutions that, in order to satisfy the risk-neutral constraints, mix inf…

• VISTA: View-Consistent Self-Verified Training for GUI Grounding

  Xinyu Qiu, Yunzhu Zhang, Heng Jia, Shuheng Shen, Changhua Meng, Linchao Zhu · 15 June 2026

  When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage. We propose VISTA (View-Consistent Self-V…

• CSPO: Constraint-Sensitive Policy Optimization for Safe Reinforcement Learning

  Ayoub Belouadah, Sylvain Kubler, Yves Le Traon · 15 June 2026

  Safe reinforcement learning (Safe RL) aims to maximize expected return while satisfying safety constraints, typically modeled as Constrained Markov Decision Processes (CMDPs). While primal-dual methods scale well to deep RL, they often suffer from delayed constraint correction, leading to oscillator…

• Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning

  Omar Adalat, Edwin Hamel-De le Court, Francesco Belardinelli · 15 June 2026

  Safe coordination problems surface in multi-agent reinforcement learning when global safety cannot be enforced by any agent unilaterally: the admissibility of one agent's action may depend on the dynamics of other agents. Decentralised shields can enforce safety at runtime, but purely factorised per…

• Provably Safe, Yet Scalable Reinforcement Learning

  Kai S. Yun, Zeyang Li, Navid Azizan · 15 June 2026

  Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but does not provide formal safety guarantees for the learned policy. In contrast, met…

• Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligenc

  Munkhdegerekh Batzorig, Purevbaatar Ganbold, Kyungbin Park, Pilkong Jeong, Kangbin Yim · 12 June 2026

  Distributed collaborative intelligence (DCI), encompassing edge-to-edge architectures, federated learning, transfer learning, and swarm systems, creates environments in which emergent risk is structurally unavoidable: locally correct decisions by individual agents compose into globally unacceptable …

• Boosting Direct Preference Optimization with Penalization

  Pengwei Sun · 12 June 2026

  Offline preference optimization has become a practical substitute for reinforcement learning from human feedback, but pairwise objectives such as Direct Preference Optimization (DPO) and its variants use only the chosen and rejected responses stored in a static dataset. This leaves a useful signal u…

• A Tutorial on World Models and Physical AI

  Il-Seok Oh · 12 June 2026

  World modeling is emerging as a central principle for building intelligent systems capable of prediction, reasoning, and decision making. A central distinction can be drawn between explicit world models, which learn structured dynamics for rollout-based reasoning and planning, and implicit world mod…

• From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence

  Zixing Lei, Genjia Liu, Yuanshuo Zhang, Qipeng Liu, Yuzhu Cai, Sixiang Chen, Jixian Wu, Yunhong Wang, Weixin Li, Chuan Wen, Bo Zhao, Shanghang Zhang, Wenzhao Lian, Siheng Chen · 12 June 2026

  The field of Embodied AI is witnessing a rapid evolution toward general-purpose robotic systems, fueled by high-fidelity simulation and large-scale data collection. However, this scaling capability remains severely bottlenecked by a reliance on labor-intensive manual oversight from intricate reward …

• WOMBET: World Model-Based Experience Transfer for Robust and Sample-efficient Reinforcement Learning

  Mintae Kim, Koushil Sreenath · 12 June 2026

  Reinforcement learning (RL) in robotics is often limited by the cost and risk of data collection, motivating experience transfer from a source task to a target task. Offline-to-online RL leverages prior data but typically assumes a given fixed dataset and does not address how to generate reliable da…

• Keep Policy Gradient in Charge: Sibling-Guided Credit Distillation for Long-Horizon Tool-Use Agents

  Tianyu Ding, Jianhong Xin, Juan Pablo De la Cruz Weinstein · 12 June 2026

  Long-horizon tool-use reinforcement learning can learn from outcome verification, but its trajectory-level advantage is broadcast across many reasoning, API, and answer tokens. Self-distillation promises a denser signal by reusing a policy's own rollouts or a privileged teacher. We show, howev…

• Select and Improve: Understanding the Mechanics of Post-Training for Reasoning

  Akshay Krishnamurthy, Audrey Huang, Nived Rajaraman · 12 June 2026

  Reinforcement learning has rapidly emerged as a key component in the training of reasoning and coding models, yet it remains poorly understood from a mechanistic perspective. We study how and through what underlying processes capabilities are acquired or enhanced via reinforcement learning post-trai…

• HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation

  Haoran Liu, Yuwei Zhang, Xiyao Li, Bohan Lyu, Jingbo Shang · 11 June 2026

  Reinforcement learning typically improves multi-turn agent capabilities through the terminal outcome of the trajectories, which makes it difficult to determine credit assignments for each intermediate turns. Recent on-policy self-distillation methods offer a promising alternative by converting privi…

• Phi-Actor-Critic: Steering General-Sum Games to Pareto-Efficient Correlated Equilibria

  Wongyu Lee, Francesco Lelli, Omran Ayoub, Massimo Tornatore · 11 June 2026

  Real-world multi-agent systems, from traffic coordination to resource allocation, are often modeled as general-sum games where individual incentives conflict with collective welfare. In these settings, the central challenge is not merely finding an equilibrium, but selecting socially desirable outco…

• UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning

  Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang · 11 June 2026

  Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the …

• Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL

  Zhuoran Li, Ling Pan, Jiatai Huang, Longbo Huang · 11 June 2026

  We present a novel Diffusion Offline Multi-agent Model (DOM2) for offline Multi-Agent Reinforcement Learning (MARL). Different from existing algorithms that rely mainly on conservatism in policy design, DOM2 enhances policy expressiveness and diversity based on diffusion model. Specifically, we inco…

• Space-sampled Value Decay: Forgetting Mechanisms for Non-stationary Deep Reinforcement Learning

  Felix St\"orck, Fabian Hinder, Barbara Hammer · 11 June 2026

  Studies on rodents such as mice have shown the capabilities to adapt their behavior when dealing with changing parameters (``drift'') of the environment even if no information about change is provided (uncertainty) -- a behavior that can be modeled by forgetting mechanisms. Non-stationary Reinforcem…

• Implicit Neural Representations of Individual Behavior

  Andrew Kang, Priya Narasimhan · 11 June 2026

  We study policy representation learning from unlabeled multi-policy behavioral data. Each episode is generated by a fixed policy, but policy labels are unavailable. This setting appears in robotics play, demonstrations, games, racing, and other datasets where heterogeneous behaviors are mixed withou…