Reinforcement Learning in Robotics

Latest papers

• ELVIS: Ensemble-Calibrated Latent Imagination for Long-Horizon Visual MPC

  Yurui Du, Pinhao Song, Yutong Hu, Renaud Detry · 7 May 2026

  A central challenge of visual control with model-based reinforcement learning (RL) is reliable long-horizon planning: long rollouts with learned latent dynamics exhibit branching futures and multi-modal action-value distributions. In addition, compounding model errors amplified by visual occlusions …

• Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime

  Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen · 7 May 2026

  SWE-bench-style agentic reinforcement learning relies on expensive stateful trajectories, yet substantial compute is wasted on sampled rollout groups with skewed pass rates, where binary rewards provide a weak contrastive signal. We frame this inefficiency as a pass-rate control problem and show tha…

• Overcoming Environmental Meta-Stationarity in MARL via Adaptive Curriculum and Counterfactual Group Advantage

  Weiqiang Jin, Yang Liu, Shixiang Tang, Jinhu Qi, Wentao Zhang, Junli Wang, Biao Zhao, Hongyang Du · 7 May 2026

  Multi-agent reinforcement learning (MARL) has reached competitive performance on cooperative tasks against scripted adversaries, yet most methods train agents at a single fixed difficulty throughout the entire run. We term this static-difficulty regime environmental meta-stationarity and show that i…

• CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies

  Keyu Chen, Nanfei Ye, Yida Wang, Wenchao Sun, Danqi Zhao, Hao Cheng, Sifa Zheng · 7 May 2026

  Open-loop imitation learning has advanced modern autonomous driving policy architectures, but closed-loop deployment remains vulnerable to policy-induced distribution shift. Existing post-training paradigms exhibit fundamental trade-offs: closed-loop RL fine-tuning provides grounded feedback from ex…

• Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning

  Shawn Ray · 7 May 2026

  System Neural Diversity (SND) measures behavioral heterogeneity in multi-agent reinforcement learning by averaging pairwise distances over all $\binom{n}{2}$ agent pairs, making each call quadratic in team size. We introduce Graph-SND, which replaces this complete-graph average with a weighted avera…

• Extending Differential Temporal Difference Methods for Episodic Problems

  Kris De Asis, Mohamed Elsayed, Jiamin He · 7 May 2026

  Differential temporal difference (TD) methods are value-based reinforcement learning algorithms that have been proposed for infinite-horizon problems. They rely on reward centering, where each reward is centered by the average reward. This keeps the return bounded and removes a value function's stat…

• EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance

  Song Yu, Li Li, Wenwen Zhao, Zhisheng Yang · 7 May 2026

  Reinforcement learning with verifiable rewards (RLVR), particularly Group Relative Policy Optimization (GRPO), has advanced LLM reasoning. However, GRPO suffers from three credit assignment failures: uniform token-level granularity that ignores heterogeneous informational value, uniform polarity tha…

• When Life Gives You BC, Make Q-functions: Extracting Q-values from Behavior Cloning for On-Robot Reinforcement Learning

  Lakshita Dodeja, Ondrej Biza, Shivam Vats, Stephen Hart, Stefanie Tellex, Robin Walters, Karl Schmeckpeper, Thomas Weng · 7 May 2026

  Behavior Cloning (BC) has emerged as a highly effective paradigm for robot learning. However, BC lacks a self-guided mechanism for online improvement after demonstrations have been collected. Existing offline-to-online learning methods often cause policies to replace previously learned good actions …

• RouteFormer: A Transformer-Based Routing Framework for Autonomous Vehicles

  Yazan Youssef, Paulo Ricardo Marques de Araujo, Aboelmagd Noureldin, Sidney Givigi · 7 May 2026

  Autonomous surveillance missions in Internet of Things (IoT) networks often involve solving NP-hard combinatorial optimization problems to ensure efficient resource utilization. To address the limitations of conventional heuristics in dynamic environments, we propose RouteFormer, a novel framework f…

• Explaining and Preventing Alignment Collapse in Iterative RLHF

  Etienne Gauthier, Francis Bach, Michael I. Jordan · 7 May 2026

  Reinforcement learning from human feedback (RLHF) typically assumes a static or non-strategic reward model (RM). In iterative deployment, however, the policy generates the data on which the RM is retrained, creating a feedback loop. Building on the Stackelberg game formulation of this interaction, w…

• Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination

  Jonathan Spieler, Sven Behnke · 7 May 2026

  State-of-the-art model-based Reinforcement Learning (RL) approaches either use gradient-free, population-based methods for planning, learned policy networks, or a combination of policy networks and planning. Hybrid approaches that combine Model Predictive Control (MPC) with a learned model and a pol…

• Using Common Random Numbers for Simulation-based Planning with Rollouts

  Sandarbh Yadav, Frederic J Maliakkal, Harshad Khadilkar, Shivaram Kalyanakrishnan · 7 May 2026

  Simulation-based planning with rollouts is a widely-deployed technique for decision making in stochastic environments. The primary instrument of simulation-based planning is a sampling model, which is repeatedly called to generate trajectories and estimate the utilities of available actions. Among t…

• Regularized Centered Emphatic Temporal Difference Learning

  Xingguo Chen, Chaohui Wu, Jinguo Ye, Chao Li, Shangdong Yang, Guang Yang, Tianyu Liang, Wenhao Wang · 7 May 2026

  Off-policy temporal-difference (TD) learning with function approximation faces a structural tradeoff among stability, projection geometry, and variance control. Emphatic TD (ETD) improves the off-policy projection geometry through follow-on emphasis, but the follow-on trace can have high variance. W…

• When Context Hurts: The Crossover Effect of Knowledge Transfer on Multi-Agent Design Exploration

  Saranyan Vigraham · 7 May 2026

  The prevailing assumption in agent orchestration is that more context is better. We test this on multi-agent software design across 10 tasks, 7 context-injection conditions, and over 2,700 runs, and find a crossover effect: the same artifact type improves design exploration on some tasks (up to 20$\…

• Structural Equivalence and Learning Dynamics in Delayed MARL

  Jules Sintes, Ana Bu\v{s}i\'c, Jiamin Zhu · 7 May 2026

  We formally establish the equivalence between Observation Delay (OD) and Action Delay (AD) in cooperative partially observable multi-agent systems using observation-action histories. We show that both systems generate identical admissible joint-policy sets, and their induced state-action-observation…

• Modular Reinforcement Learning For Cooperative Swarms

  Erel Shtossel, Gal A. Kaminka · 7 May 2026

  A cooperative robot swarm is a collective of computationally-limited robots that share a common goal. Each robot can only interact with a small subset of its peers, without knowing how this affects the collective utility. Recent advances in distributed multi-agent reinforcement learning have demonst…

• LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts

  Seungeun Rho, Shamel Fahmi, Jeonghwan Kim, Arianna Ilvonen, Sehoon Ha, Gabriel Nelson · 7 May 2026

  Designing reward functions for agile robotic maneuvers in reinforcement learning remains difficult, and demonstration-based approaches often require reference motions that are unavailable for novel platforms or extreme stunts. We present LineRides, a line-guided learning framework that enables a cus…

• Adaptive Policy Selection and Fine-Tuning under Interaction Budgets for Offline-to-Online Reinforcement Learning

  Alper Kamil Bozkurt, Xiaoan Xu, Shangtong Zhang, Miroslav Pajic, Yuichi Motai · 7 May 2026

  In offline-to-online reinforcement learning (O2O-RL), policies are first safely trained offline using previously collected datasets and then further fine-tuned for tasks via limited online interactions. In a typical O2O-RL pipeline, candidate policies trained with offline RL are evaluated via either…

• Constraint-Enhanced Reinforcement Learning Based on Dynamic Decoupled Spherical Radial Squashing

  Qijun Liao, Zhaoxin Yu, Jue Yang · 7 May 2026

  When deploying reinforcement learning policies to physical robots, actuator rate constraints -- hard limits on how fast each joint can move per control step -- are unavoidable. These limits vary substantially across joints due to differences in motor inertia, power bandwidth, and transmission stiffn…

• A Brief Overview: Agentic Reinforcement Learning In Large Language Models

  Fangming Cui, Ruixiao Zhu, Cheng Fang, Sunan Li, Jiahong Li · 7 May 2026

  Reinforcement Learning (RL) has traditionally focused on training specialized agents to optimize predefined reward functions within narrowly defined environments. However, the advent of powerful Large Language Models (LLMs) and increasingly complex, open-ended tasks has catalyzed a paradigm shift to…

• The Implicit Curriculum: Learning Dynamics in RL with Verifiable Rewards

  Yu Huang, Zixin Wen, Yuejie Chi, Yuting Wei, Aarti Singh, Yingbin Liang, Yuxin Chen · 7 May 2026

  Reinforcement learning with verifiable rewards (RLVR) has been a main driver of recent breakthroughs in large reasoning models. Yet it remains a mystery how rewards based solely on final outcomes can help overcome the long-horizon barrier to extended reasoning. To understand this, we develop a theor…

• Hierarchical Support Vector State Partitioning for Distilling Black Box Reinforcement Learning Policies

  Senne Deproost, Mehrdad Asadi, Ann Now\'e · 7 May 2026

  We introduce State Vector Space Partitioning (SVSP), a novel method to mimic a black box reinforcement learning policy using a set of human-interpretable subpolicies. By partitioning a distillation dataset of state action pairs with linear support vector machine splits, SVSP constructs a compact and…

• Data-dependent Exploration for Online Reinforcement Learning from Human Feedback

  Zhen-Yu Zhang, Yuting Tang, Jiandong Zhang, Lanjihong Ma, Masashi Sugiyama · 7 May 2026

  Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for aligning large language models (LLMs) by continuously collecting new preference feedback during training. A foundational challenge in this setting is exploration, which requires algorithms that enable th…

• A Harmonic Mean Formulation of Average Reward Reinforcement Learning in SMDPs

  Erel Shtossel, Alicia Vidler, Uri Shaham, Gal A. Kaminka · 7 May 2026

  Recent research has revived and amplified interest in algorithms for undiscounted average reward reinforcement learning in infinite-horizon, non-episodic (continuing) tasks. Semi-Markov decision processes (SMDPs) are of particular interest. In SMDPs, discrete actions stochastically generate both rew…

• Free Energy-Driven Reinforcement Learning with Adaptive Advantage Shaping for Unsupervised Reasoning in LLMs

  Yiming Huang, Zhenbo Shi, Xin-Cheng Wen, Jichuan Zeng, Cuiyun Gao, Peiyi Han, Chuanyi Liu · 7 May 2026

  Unsupervised reinforcement learning (RL) has emerged as a promising paradigm for enabling self-improvement in large language models (LLMs). However, existing unsupervised RL-based methods often lack the capacity to adapt to the model's evolving reasoning capabilities during training. Therefore, thes…