Reinforcement Learning in Robotics

Latest papers

• SMAC: Score-Matched Actor-Critics for Robust Offline-to-Online Transfer

  Nathan S. de Lara, Florian Shkurti · 20 February 2026

  Modern offline Reinforcement Learning (RL) methods find performant actor-critics, however, fine-tuning these actor-critics online with value-based RL algorithms typically causes immediate drops in performance. We provide evidence consistent with the hypothesis that, in the loss landscape, offline ma…

• Training Large Reasoning Models Efficiently via Progressive Thought Encoding

  Zeliang Zhang, Xiaodong Liu, Hao Cheng, Hao Sun, Chenliang Xu, Jianfeng Gao · 20 February 2026

  Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rewards, where autoregressive decoding dominates time and memory usage. While sliding-window cache strategies can bound mem…

• A Unified Framework for Locality in Scalable MARL

  Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni, Lijun Chen · 20 February 2026

  Scalable Multi-Agent Reinforcement Learning (MARL) is fundamentally challenged by the curse of dimensionality. A common solution is to exploit locality, which hinges on an Exponential Decay Property (EDP) of the value function. However, existing conditions that guarantee the EDP are often conservati…

• EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments

  Sushant Mehta, Logan Ritchie, Suhaas Garre, Nick Heiner, Edwin Chen · 20 February 2026

  We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce CoreCraft, the first environment in EnterpriseBench, Surge AI's suite of agentic RL environments. CoreCraft is a fully operational…

• MDP Planning as Policy Inference

  David Tolpin · 20 February 2026

  We cast episodic Markov decision process (MDP) planning as Bayesian inference over _policies_. A policy is treated as the latent variable and is assigned an unnormalized probability of optimality that is monotone in its expected return, yielding a posterior distribution whose modes coincide with ret…

• Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning

  Sanghyeon Lee, Sangjun Bae, Yisak Park, Seungyul Han · 20 February 2026

  Meta-reinforcement learning (Meta-RL) facilitates rapid adaptation to unseen tasks but faces challenges in long-horizon environments. Skill-based approaches tackle this by decomposing state-action sequences into reusable skills and employing hierarchical decision-making. However, these methods are h…

• Phase-Aware Mixture of Experts for Agentic Reinforcement Learning

  Shengtian Yang (Southeast University, Kuaishou Technology), Yu Li (Southeast University), Shuo He (Nanyang Technological University), Yewen Li (Kuaishou Technology), Qingpeng Cai (Kuaishou Technology), Peng Jiang (Kuaishou Technology), Lei Feng (Southeast University) · 20 February 2026

  Reinforcement learning (RL) has equipped LLM agents with a strong ability to solve complex tasks. However, existing RL methods normally use a \emph{single} policy network, causing \emph{simplicity bias} where simple tasks occupy most parameters and dominate gradient updates, leaving insufficient cap…

• Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learning

  Yonghyeon Jo, Sunwoo Lee, Seungyul Han · 20 February 2026

  Value decomposition is a core approach for cooperative multi-agent reinforcement learning (MARL). However, existing methods still rely on a single optimal action and struggle to adapt when the underlying value function shifts during training, often converging to suboptimal policies. To address this …

• Online Robust Reinforcement Learning with General Function Approximation

  Debamita Ghosh, George K. Atia, Yue Wang · 20 February 2026

  In many real-world settings, reinforcement learning systems suffer performance degradation when the environment encountered at deployment differs from that observed during training. Distributionally robust reinforcement learning (DR-RL) mitigates this issue by seeking policies that maximize performa…

• Continual uncertainty learning

  Heisei Yonezawa, Ansei Yonezawa, Itsuro Kajiwara · 20 February 2026

  Robust control of mechanical systems with multiple uncertainties remains a fundamental challenge, particularly when nonlinear dynamics and operating-condition variations are intricately intertwined. While deep reinforcement learning (DRL) combined with domain randomization has shown promise in mitig…

• Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning

  Nikunj Gupta, James Zachary Hare, Jesse Milzman, Rajgopal Kannan, Viktor Prasanna · 20 February 2026

  Coordinating actions is the most fundamental form of cooperation in multi-agent reinforcement learning (MARL). Successful decentralized decision-making often depends not only on good individual actions, but on selecting compatible actions across agents to synchronize behavior, avoid conflicts, and s…

• Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learning

  Jaebak Hwang, Sanghyeon Lee, Jeongmo Kim, Seungyul Han · 20 February 2026

  Long-horizon goal-conditioned tasks pose fundamental challenges for reinforcement learning (RL), particularly when goals are distant and rewards are sparse. While hierarchical and graph-based methods offer partial solutions, their reliance on conventional hindsight relabeling often fails to correct …

• Discovering Multiagent Learning Algorithms with Large Language Models

  Zun Li, John Schultz, Daniel Hennes, Marc Lanctot · 20 February 2026

  Much of the advancement of Multi-Agent Reinforcement Learning (MARL) in imperfect-information games has historically depended on manual iterative refinement of baselines. While foundational families like Counterfactual Regret Minimization (CFR) and Policy Space Response Oracles (PSRO) rest on solid …

• Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning

  Xuefeng Wang, Lei Zhang, Henglin Pu, Ahmed H. Qureshi, Husheng Li · 20 February 2026

  Existing reinforcement learning (RL) methods struggle with complex dynamical systems that demand interactions at high frequencies or irregular time intervals. Continuous-time RL (CTRL) has emerged as a promising alternative by replacing discrete-time Bellman recursion with differential value functio…

• MASPO: Unifying Gradient Utilization, Probability Mass, and Signal Reliability for Robust and Sample-Efficient LLM Reasoning

  Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Binbin Zheng, Chaowen Hu, Zekai Shao, Cong Qin, Lu Pan, Ke Zeng, Xunliang Cai · 20 February 2026

  Existing Reinforcement Learning with Verifiable Rewards (RLVR) algorithms, such as GRPO, rely on rigid, uniform, and symmetric trust region mechanisms that are fundamentally misaligned with the complex optimization dynamics of Large Language Models (LLMs). In this paper, we identify three critical c…

• Linear Convergence in Games with Delayed Feedback via Extra Prediction

  Yuma Fujimoto, Kenshi Abe, Kaito Ariu · 20 February 2026

  Feedback delays are inevitable in real-world multi-agent learning. They are known to severely degrade performance, and the convergence rate under delayed feedback is still unclear, even for bilinear games. This paper derives the rate of linear convergence of Weighted Optimistic Gradient Descent-Asce…

• Learning Distributed Equilibria in Linear-Quadratic Stochastic Differential Games: An $\alpha$-Potential Approach

  Philipp Plank, Yufei Zhang · 19 February 2026

  We analyze independent policy-gradient (PG) learning in $N$-player linear-quadratic (LQ) stochastic differential games. Each player employs a distributed policy that depends only on its own state and updates the policy independently using the gradient of its own objective. We establish global linear…

• ModalImmune: Immunity Driven Unlearning via Self Destructive Training

  Rong Fu, Jia Yee Tan, Wenxin Zhang, Zijian Zhang, Ziming Wang, Zhaolu Kang, Muge Qi, Shuning Zhang, Simon Fong · 19 February 2026

  Multimodal systems are vulnerable to partial or complete loss of input channels at deployment, which undermines reliability in real-world settings. This paper presents ModalImmune, a training framework that enforces modality immunity by intentionally and controllably collapsing selected modality inf…

• Improved Bounds for Reward-Agnostic and Reward-Free Exploration

  Oran Ridel, Alon Cohen · 19 February 2026

  We study reward-free and reward-agnostic exploration in episodic finite-horizon Markov decision processes (MDPs), where an agent explores an unknown environment without observing external rewards. Reward-free exploration aims to enable $\epsilon$-optimal policies for any reward revealed after explor…

• Multi-agent cooperation through in-context co-player inference

  Marissa A. Weis, Maciej Wo{\l}czyk, Rajai Nasser, Rif A. Saurous, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento, Alexander Meulemans · 19 February 2026

  Achieving cooperation among self-interested agents remains a fundamental challenge in multi-agent reinforcement learning. Recent work showed that mutual cooperation can be induced between "learning-aware" agents that account for and shape the learning dynamics of their co-players. However, existing …

• EnterpriseGym Corecraft: Training Generalizable Agents on High-Fidelity RL Environments

  Sushant Mehta, Logan Ritchie, Suhaas Garre, Nick Heiner, Edwin Chen · 19 February 2026

  We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce \corecraft{}, the first environment in \textsc{EnterpriseGym}, Surge AI's suite of agentic RL environments. \corecraft{} is a full…

• Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning

  Emile Anand, Richard Hoffmann, Sarah Liaw, Adam Wierman · 19 February 2026

  Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mean-field methods alleviate this burden by aggregating agent interactions, but the…

• Almost Sure Convergence of Differential Temporal Difference Learning for Average Reward Markov Decision Processes

  Ethan Blaser, Jiuqi Wang, Shangtong Zhang · 19 February 2026

  The average reward is a fundamental performance metric in reinforcement learning (RL) focusing on the long-run performance of an agent. Differential temporal difference (TD) learning algorithms are a major advance for average reward RL as they provide an efficient online method to learn the value fu…

• HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents

  Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou, Charles Fleming, Zhaoran Wang, Alfredo Garcia, Mingyi Hong · 19 February 2026

  Training LLMs as interactive agents for multi-turn decision-making remains challenging, particularly in long-horizon tasks with sparse and delayed rewards, where agents must execute extended sequences of actions before receiving meaningful feedback. Most existing reinforcement learning (RL) approach…

• Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learning

  Oswin So, Eric Yang Yu, Songyuan Zhang, Matthew Cleaveland, Mitchell Black, Chuchu Fan · 18 February 2026

  Recent advances in deep reinforcement learning (RL) have achieved strong results on high-dimensional control tasks, but applying RL to reachability problems raises a fundamental mismatch: reachability seeks to maximize the set of states from which a system remains safe indefinitely, while RL optimiz…