Reinforcement Learning in Robotics

Neueste Paper

• Towards Learning Representations of Policies in Two-Player Zero-Sum Imperfect-Information Games

  Kevin Wang, Kevin Yang, Arjun Prakash, Amy Greenwald · 3. Juli 2026

  We investigate the problem of learning useful policy representations (embeddings) in two-player zero-sum imperfect-information games. We make three contributions: First, we introduce methods of creating datasets of policies for a given game. Second, we propose methods to learn policy representations…

• Full Bayesian Reinforcement Learning via LF-IBIS

  Stefano Masini, Cecilia Viscardi, Michela Baccini · 3. Juli 2026

  Reinforcement Learning (RL) is a sequential decision-making framework in which an agent learns optimal policies through interaction with an environment by maximizing cumulative rewards. Among RL methods, Bayesian Reinforcement Learning (BRL) addresses common practical challenges related to data scar…

• Rank-Then-Act: Reward-Free Control from Frame-Order Progress

  Yuriy Maksyuta, George Bredis, Ruslan Rakhimov, Daniil Gavrilov · 3. Juli 2026

  We introduce Rank-Then-Act (RTA), a framework for learning control policies from expert video demonstrations without environment rewards. RTA trains a Vision-Language Model (VLM) offline as a progress-based ordinal scorer, using a Group Relative Policy Optimization (GRPO) objective over shuffled fra…

• BIFROST: Bridging Invariant Feature Representation for Observation-space Sim2Real Transfer

  Yunfu Deng, Josiah P. Hanna · 3. Juli 2026

  Sim2real transfer for robot policy learning suffers due to mismatch between simulation and reality. Existing methods typically address each gap in isolation through separate adaptation modules, which are composed or layered when both gaps coexist. Yet the basis for attempting sim2real in the first p…

• Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probes

  Xue Qin, Simin Luan, Cong Yang, Zhijun Li · 3. Juli 2026

  Compositional machine-learning (ML) systems assemble runtime behavior from libraries of independently re-trained capability modules. Replacing one module raises a regression-testing question that static dependence analysis cannot answer: which existing compositions stay valid, and at what test cost?…

• Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

  Shenghui Zhang, YuXuan Gao, Songwei Zhao, Jifeng Hu, Zijing Zhang, Hechang Chen · 3. Juli 2026

  With the rapid development of autonomous aerial systems, Unmanned Aerial Vehicles (UAVs) are increasingly deployed in applications such as inspection, environmental monitoring, and rescue, creating growing demand for reliable autonomous navigation. However, autonomous UAV navigation in dense environ…

• WorldSample: Closed-loop Real-robot RL with World Modelling

  Yuquan Xue, Le Xu, Zeyi Liu, Zhenyu Wu, Zhengyi Gu, Xinyang Song, Bofang Jia, Ziwei Wang · 3. Juli 2026

  Reinforcement learning (RL) can overcome the demonstration-coverage limitation of imitation learning (IL) by allowing robots to improve through trial-and-error interaction beyond the states observed in demonstrations. However, deploying RL on real robots remains constrained by high interaction costs…

• SAGA: Scene-Aware, Goal-Evolving Agents for Long-Horizon CivRealm Strategy Planning

  Tianyu Jin, Shuo Chen, Yida Wang, Liuyu Xiang, Yingzhuo Liu, Zhiyao Jiang, Yexin Li, Zhaofeng He · 3. Juli 2026

  Long-horizon strategic planning in complex strategy games demands concurrent reasoning across multiple decision domains under imperfect information and sparse reward. Existing LLM-based agents suffer from three systematic failures: scene blindness from raw tile coordinates, context overflow and doma…

• Cross-Platform Control for Autonomous Surface Vehicles via Adaptive Reinforcement Learning

  Ruiheng Jiang, Thomas Bi, Raffaello D'Andrea, Aswin Ramachandran · 3. Juli 2026

  Autonomous surface vehicles vary widely in hydrodynamic and actuation characteristics, yet most controllers are designed for single-platform deployment. We present an adaptive reinforcement learning approach for trajectory tracking that enables zero-shot cross-platform deployment using a single poli…

• Evidence-State Rewards for Long-Context Reasoning

  Ya Gao, Pekka Marttinen · 3. Juli 2026

  Long-context reasoning requires models to locate, revise, and synthesize evidence distributed across lengthy inputs. Existing long-context RL methods usually reward final answers or static evidence extraction, offering little feedback on how intermediate actions change the model's evidence state. We…

• Learning the Supports for Categorical Critic in Reinforcement Learning

  Jen-Yen Chang, Takayuki Osa, Tatsuya Harada · 3. Juli 2026

  Value functions are an essential component in actor-critic based deep reinforcement learning (RL). Conventionally, these functions are trained as a regression task by minimising the mean squared error (MSE) relative to bootstrapped target values. Meanwhile, in distributional RL, a distribution of re…

• Generalization in offline RL: The structure is more important than the amount of pessimism

  Max Weltevrede, Matthijs T. J. Spaan, Wendelin B\"ohmer · 3. Juli 2026

  While pessimism counteracts overestimation bias in offline reinforcement learning (RL), being overly conservative has been associated with hindering certain forms of generalization. However, in this paper we demonstrate that being overly pessimistic does not inherently prevent optimal generalization…

• EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

  Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang · 3. Juli 2026

  Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting…

• Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL

  Juliette Decugis, Sean O'Brien, Francis Bach, Gabriel Synnaeve, Taco Cohen · 3. Juli 2026

  Reinforcement learning post-training dramatically improves LLM reasoning, but suffers from training instability and diversity collapse. Advantage functions offer an appealing fix: they reshape the training objective, reweight which rollouts drive learning, and are trivial to implement. Yet a prolife…

• Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL

  Juliette Decugis, Sean O'Brien, Francis Bach, Gabriel Synnaeve, Taco Cohen · 3. Juli 2026

  Reinforcement learning post-training dramatically improves LLM reasoning, but suffers from training instability and diversity collapse. Advantage functions offer an appealing fix: they reshape the training objective, reweight which rollouts drive learning, and are trivial to implement. Yet a prolife…

• Local Motion Matters: A Deconstruct-Recompose Paradigm for Reinforcement Learning Pre-training from Videos

  Jinwen Wang, Youfang Lin, Xiaobo Hu, Shuo Wang, Kai Lv · 2. Juli 2026

  Pre-training on large-scale videos to improve reinforcement learning efficiency is promising yet remains challenging. Existing methods typically treat the agent as an indivisible entity, modeling motion patterns globally. Such global modeling is tightly coupled with the morphology, hindering transfe…

• HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning

  Runze Zhao, Dongruo Zhou, Sumit Kumar Jha, Nathaniel D. Bastian, Ankit Shah · 2. Juli 2026

  Cooperative multi-agent reinforcement learning (MARL) often relies on communication to mitigate partial observability, yet most existing protocols treat messages as flat dense vectors detached from the structure of the observations they summarize. This design overlooks an important source of inducti…

• Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligence

  Munkhdelgerekh Batzorig, Purevbaatar Ganbold, Kyungbin Park, Pilkong Jeong, Kangbin Yim · 2. Juli 2026

  Distributed collaborative intelligence (DCI), encompassing edge-to-edge architectures, federated learning, transfer learning, and swarm systems, creates environments in which emergent risk is structurally unavoidable: locally correct decisions by individual agents compose into globally unacceptable …

• Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalization

  Jinwen Wang, Youfang Lin, Xiaobo Hu, Qian Xu, Shuo Wang, Zhuo Chen, Kai Lv · 2. Juli 2026

  Visual Reinforcement Learning (VRL) has achieved considerable success in solving control tasks. However, generalizing learned policies to new environments remains a major challenge, as agents often overfit to task-irrelevant features in the training environment. To solve this problem, we introduce t…

• Multi-scale Mixture of World Models for Embodied Agents in Evolving Environments

  Jinwoo Jang, Daniel J. Rho, Sihyung Yoon, Hyunsuk Cho, Honguk Woo · 2. Juli 2026

  Embodied agents operating in the real world require multi-scale reasoning and knowledge adaptation as conditions change. We identify two challenges in applying Mixture of Experts (MoE) to this setting: routing lacks an explicit notion of scale, preventing targeted updates at specific scales, and a u…

• Self-Evolving Agents with Anytime-Valid Certificates

  Biswa Sengupta · 2. Juli 2026

  Self-evolving agents violate the assumption behind most learning-theoretic guarantees: the data, evaluator, components, and hypothesis space are produced by the policy being updated. We present \textbf{SEA}, an architecture that confines self-modification to a small steering adapter and a versioned …

• Gauging, Measuring, and Controlling Critic Complexity in Actor-Critic Reinforcement Learning

  Konstantin Garbers · 2. Juli 2026

  Actor-critic methods depend on learned critics, but critic quality is often evaluated only indirectly through return, temporal-difference error, or value loss. Critic complexity is introduced as an additional diagnostic and intervention dimension for actor-critic reinforcement learning. The analysis…

• Language-Critique Imitation Learning from Suboptimal Demonstrations

  Chih-Han Yang, Dai-Jie Wu, Yun-Ping Huang, Ping-Chun Hsieh, Kenneth Marino, Shao-Hua Sun · 2. Juli 2026

  Prior work on imitation learning from suboptimal demonstrations typically relies on compressed supervision signals such as confidence estimates, discriminator scores, or importance weights. These scalar signals are inherently limited, as they cannot explicitly express intermediate reasoning about ta…

• Valdi: Value Diffusion World Models

  Christopher Lindenberg, Kashyap Chitta · 2. Juli 2026

  World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive enough to represent uncertain futures. Diffusion models offer a natural mechanism for modeling uncertain dynamics, yet their iterative inference proced…

• Staleness-Learning Rate Scaling Laws for Asynchronous RLHF

  Jingwei Song, Haofeng Xu, Jie Xiao, Chengke Bao, Jingwei Shi, Pengbin Feng, Weixun Wang, Yuhang Han, Chuan Wu, Linfeng Zhang, Bill Shi · 2. Juli 2026

  High-throughput RLHF systems often decouple rollout generation from policy optimization, leading to the use of stale rollouts during learner updates. In this work, we study the effect of such staleness in asynchronous GRPO. We make the behavior policy explicit in the GRPO surrogate objective and dis…