Reinforcement Learning in Robotics

Latest papers

• Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking

  Disha Singha · 30 April 2026

  Reinforcement learning (RL) systems typically optimize scalar reward functions that assume precise and reliable evaluation of outcomes. However, real-world objectives--especially those derived from human preferences--are often uncertain, context-dependent, and internally inconsistent. This mismatch …

• Rule-based High-Level Coaching for Goal-Conditioned Reinforcement Learning in Search-and-Rescue UAV Missions Under Limited-Simulation Training

  Mahya Ramezani, Holger Voos · 30 April 2026

  This paper presents a hierarchical decision-making framework for unmanned aerial vehicle (UAV) missions motivated by search-and-rescue (SAR) scenarios under limited simulation training. The framework combines a fixed rule-based high-level advisor with an online goal-conditioned low-level reinforceme…

• Lyapunov-Guided Self-Alignment: Test-Time Adaptation for Offline Safe Reinforcement Learning

  Seungyub Han, Hyungjin Kim, Jungwoo Lee · 30 April 2026

  Offline reinforcement learning (RL) agents often fail when deployed, as the gap between training datasets and real environments leads to unsafe behavior. To address this, we present SAS (Self-Alignment for Safety), a transformer-based framework that enables test-time adaptation in offline safe RL wi…

• Atomic-Probe Governance for Skill Updates in Compositional Robot Policies

  Xue Qin, Simin Luan, John See, Cong Yang, Zhijun Li · 30 April 2026

  Skill libraries in deployed robotic systems are continually updated through fine-tuning, fresh demonstrations, or domain adaptation, yet existing typed-composition methods (BLADE, SymSkill, Generative Skill Chaining) treat the library as frozen at test time and do not analyze how composition outcome…

• Adaptive Scaling of Policy Constraints for Offline Reinforcement Learning

  Tan Jing, Xiaorui Li, Chao Yao, Xiaojuan Ban, Yuetong Fang, Renjing Xu, Zhaolin Yuan · 30 April 2026

  Offline reinforcement learning (RL) enables learning effective policies from fixed datasets without any environment interaction. Existing methods typically employ policy constraints to mitigate the distribution shift encountered during offline RL training. However, because the scale of the constrain…

• DC-Ada: Reward-Only Decentralized Sensor Adaptation for Heterogeneous Multi-Robot Teams

  Saad Alqithami · 30 April 2026

  Heterogeneity is a defining feature of deployed multi-robot teams: platforms often differ in sensing modalities, ranges, fields of view, and failure patterns. Controllers trained under nominal sensing can degrade sharply when deployed on robots with missing or mismatched sensors, even when the task …

• A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety

  Ankita Kushwaha, Kiran Ravish, Preeti Lamba, Pawan Kumar · 30 April 2026

  Safe Reinforcement Learning (SafeRL) is the subfield of reinforcement learning that explicitly deals with safety constraints during the learning and deployment of agents. This survey provides a mathematically rigorous overview of SafeRL formulations based on Constrained Markov Decision Processes (CM…

• Near-Optimal Sample Complexities of Divergence-based S-rectangular Distributionally Robust Reinforcement Learning

  Zhenghao Li, Shengbo Wang, Nian Si · 29 April 2026

  Distributionally robust reinforcement learning (DR-RL) has recently gained significant attention as a principled approach that addresses discrepancies between training and testing environments. To balance robustness, conservatism, and computational traceability, the literature has introduced DR-RL m…

• SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models

  Yuxuan Jiang, Francis Ferraro · 29 April 2026

  Training reliable tool-augmented agents remains a significant challenge, largely due to the difficulty of credit assignment in multi-step reasoning. While process-level reward models offer a promising direction, existing LLM-based judges often produce noisy and inconsistent signals because they lack…

• TSN-Affinity: Similarity-Driven Parameter Reuse for Continual Offline Reinforcement Learning

  Dominik \.Zurek, Kamil Faber, Marcin Pietron, Pawe{\l} Gajewski, Roberto Corizzo · 29 April 2026

  Continual offline reinforcement learning (CORL) aims to learn a sequence of tasks from datasets collected over time while preserving performance on previously learned tasks. This setting corresponds to domains where new tasks arise over time, but adapting the model in live environment interactions i…

• Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings

  Keenan Powell, Peihong Yu, Pratap Tokekar · 29 April 2026

  Many Multi-Agent Reinforcement Learning (MARL) agents fail to adapt properly to cooperating with agents trained with the same objectives but different seeds, algorithms, or other training differences. This is the problem of Zero-Shot Coordination (ZSC), which focuses on training agents to cooperate …

• DLM: Unified Decision Language Models for Offline Multi-Agent Sequential Decision Making

  Zhuohui Zhang, Bin Cheng, Bin He · 29 April 2026

  Building scalable and reusable multi-agent decision policies from offline datasets remains a challenge in offline multi-agent reinforcement learning (MARL), as existing methods often rely on fixed observation formats and action spaces that limit generalization. In contrast, large language models (LL…

• Safe-Support Q-Learning: Learning without Unsafe Exploration

  Yeeun Lim, Narim Jeong, Donghwan Lee · 29 April 2026

  Ensuring safety during reinforcement learning (RL) training is critical in real-world applications where unsafe exploration can lead to devastating outcomes. While most safe RL methods mitigate risk through constraints or penalization, they still allow exploration of unsafe states during training. I…

• SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning

  Zijian Guo, \.Ilker I\c{s}{\i}k, H. M. Sabbir Ahmad, Wenchao Li · 28 April 2026

  Specification-guided reinforcement learning (RL) provides a principled framework for encoding complex, temporally extended tasks using formal specifications such as linear temporal logic (LTL). While recent methods have shown promising results, their ability to generalize across unseen specification…

• TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents

  Jiaqi Wang, Wenhao Zhang, Weijie Shi, Yaliang Li, James Cheng · 28 April 2026

  On-policy distillation (OPD) has shown strong potential for transferring reasoning ability from frontier or domain-specific models to smaller students. While effective on static single-turn tasks, its behavior in multi-turn agent settings remains underexplored. In this work, we identify a key limita…

• CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning

  Marcel Hedman, Kale-ab Abebe Tessera, Juan Claude Formanek, Anya Sims, Riccardo Zamboni, Trevor McInroe, John Torr, Elliot Fosong · 28 April 2026

  Offline multi-agent reinforcement learning (MARL) enables policy learning from fixed datasets, but is prone to coordination failure: agents trained on static, off-policy data converge to suboptimal joint behaviours because they cannot co-adapt as their policies change. We introduce CODA (Coordinatio…

• Leveraging Human Feedback for Semantically-Relevant Skill Discovery

  Maxence Hussonnois, Thommen George Karimpanal, Santu Rana · 28 April 2026

  Unsupervised skill discovery in reinforcement learning aims to intrinsically motivate agents to discover diverse and useful behaviours. However, unconstrained approaches can produce unsafe, unethical, or misaligned behaviours. To mitigate these risks and improve the practical desireability of discov…

• Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance

  Xinzhu Chen, Wei He, Huichuan Fan, Wenzhe Niu, Zhongxiang Sun, Xuanru Wang, Jiuchong Gao, Jinghua Hao, Renqing He, Weijie Yu · 28 April 2026

  Group Relative Policy Optimization (GRPO) performs coarse-grained credit assignment in reinforcement learning with verifiable rewards (RLVR) by assigning the same advantage to all tokens in a rollout. Process reward models can provide finer-grained supervision, but they require step-level annotation…

• GIFT: Global stabilisation via Intrinsic Fine Tuning

  Rory Young, Nicolas Pugeault · 28 April 2026

  Deep reinforcement learning policies achieve strong performance in complex continuous control environments with nonlinear contact forces. However, these policies often produce chaotic state dynamics, with trivially small changes to the initial conditions significantly impacting the long-term behavio…

• K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning

  Zixuan Xia, Quanxi Li · 28 April 2026

  We propose a simple yet effective alternative to reward normalization in policy gradient reinforcement learning by integrating a 1D Kalman filter for online reward estimation. Instead of relying on fixed heuristics, our method recursively estimates the latent reward mean, smoothing high-variance ret…

• Hierarchical Behaviour Spaces

  Michael Tryfan Matthews, Anssi Kanervisto, Jakob Foerster, Pierluca D'Oro, Scott Fujimoto, Mikael Henaff · 28 April 2026

  Recent work in hierarchical reinforcement learning has shown success in scaling to billions of timesteps when learning over a set of predefined option reward functions. We show that, instead of using a single reward function per option, the reward functions can be effectively used to induce a space …

• Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence

  Donghwan Lee, Hyukjun Yang · 28 April 2026

  Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic programming, this approach has received relatively less attention, main…

• When Policies Cannot Be Retrained: A Unified Closed-Form View of Post-Training Steering in Offline Reinforcement Learning

  Elias Hossain, Mohammad Jahid Ibna Basher, Ivan Garibay, Ozlem Garibay, Niloofar Yousefi · 28 April 2026

  Offline reinforcement learning (RL) can learn effective policies from fixed datasets, but deployment objectives may change after training, and in many applications the trained actor cannot be retrained because of data, cost, or governance constraints. We study deployment-time adaptation for frozen o…

• A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning

  Ying-Tu Chen, Wei Hung, Bing-Shu Wu, Zhang-Wei Hong, Ping-Chun Hsieh · 28 April 2026

  Many sequential decision-making tasks involve optimizing multiple conflicting objectives, requiring policies that adapt to different user preferences. In multi-objective reinforcement learning (MORL), one widely studied approach} addresses this by training a single policy network conditioned on pref…

• Secondary Bounded Rationality: A Theory of How Algorithms Reproduce Structural Inequality in AI Hiring

  Jia Xiao · 27 April 2026

  AI-driven recruitment systems, while promising efficiency and objectivity, often perpetuate systemic inequalities by encoding cultural and social capital disparities into algorithmic decision making. This article develops and defends a novel theory of secondary bounded rationality, arguing that AI s…