Reinforcement Learning in Robotics

Latest papers

• An Actor-Critic Framework for Continuous-Time Jump-Diffusion Controls with Normalizing Flows

  Liya Guo, Ruimeng Hu, Xu Yang, Yi Zhu · 8 April 2026

  Continuous-time stochastic control with time-inhomogeneous jump-diffusion dynamics is central in finance and economics, but computing optimal policies is difficult under explicit time dependence, discontinuous shocks, and high dimensionality. We propose an actor-critic framework that serves as a mes…

• Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner

  Andrei Polubarov, Lyubaykin Nikita, Alexander Derevyagin, Artyom Grishin, Igor Saprygin, Aleksandr Serkov, Mark Averchenko, Daniil Tikhonov, Maksim Zhdanov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Alexey Zemtsov, Vladislav Kurenkov · 8 April 2026

  Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorithm Distillation (AD) pioneered this paradigm and was subsequently scaled to multi-domain settings, although its ability t…

• A Survey of Continual Reinforcement Learning

  Chaofan Pan, Xin Yang, Yanhua Li, Wei Wei, Tianrui Li, Bo An, Jiye Liang · 8 April 2026

  Reinforcement Learning (RL) is an important machine learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in this field due to the rapid development of deep neural networks. However, the success of RL currently relies on extensive training…

• Value Mirror Descent for Reinforcement Learning

  Zhichao Jia, Guanghui Lan · 8 April 2026

  Value iteration-type methods have been extensively studied for computing a nearly optimal value function in reinforcement learning (RL). Under a generative sampling model, these methods can achieve sharper sample complexity than policy optimization approaches, particularly in their dependence on the…

• Territory Paint Wars: Diagnosing and Mitigating Failure Modes in Competitive Multi-Agent PPO

  Diyansha Singh · 8 April 2026

  We present Territory Paint Wars, a minimal competitive multi-agent reinforcement learning environment implemented in Unity, and use it to systematically investigate failure modes of Proximal Policy Optimisation (PPO) under self-play. A first agent trained for $84{,}000$ episodes achieves only $26.8\…

• ACE-Bench: Agent Configurable Evaluation with Scalable Horizons and Controllable Difficulty under Lightweight Environments

  Wang Yang, Chaoda Song, Xinpeng Li, Debargha Ganguly, Chuang Ma, Shouren Wang, Zhihao Dou, Yuli Zhou, Vipin Chaudhary, Xiaotian Han · 8 April 2026

  Existing Agent benchmarks suffer from two critical limitations: high environment interaction overhead (up to 41\% of total evaluation time) and imbalanced task horizon and difficulty distributions that make aggregate scores unreliable. To address these issues, we propose ACE-Bench built around a uni…

• Scalar Federated Learning for Linear Quadratic Regulator

  Mohammadreza Rostami, Shahriar Talebi, Solmaz S. Kia · 8 April 2026

  We propose ScalarFedLQR, a communication-efficient federated algorithm for model-free learning of a common policy in linear quadratic regulator (LQR) control of heterogeneous agents. The method builds on a decomposed projected gradient mechanism, in which each agent communicates only a scalar projec…

• Reasoning Through Chess: How Reasoning Evolves from Data Through Fine-Tuning and Reinforcement Learning

  Lucas Dionisopoulos, Nicklas Majamaki, Prithviraj Ammanabrolu · 8 April 2026

  How can you get a language model to reason in a task it natively struggles with? We study how reasoning evolves in a language model -- from supervised fine-tuning (SFT) to reinforcement learning (RL) -- by analyzing how a set of theoretically-inspired datasets impacts language model performance in c…

• MARL-GPT: Foundation Model for Multi-Agent Reinforcement Learning

  Maria Nesterova, Mikhail Kolosov, Anton Andreychuk, Egor Cherepanov, Oleg Bulichev, Alexey Kovalev, Konstantin Yakovlev, Aleksandr Panov, Alexey Skrynnik · 8 April 2026

  Recent advances in multi-agent reinforcement learning (MARL) have demonstrated success in numerous challenging domains and environments, but typically require specialized models for each task. In this work, we propose a coherent methodology that makes it possible for a single GPT-based model to lear…

• DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search

  Fang Wu, Weihao Xuan, Heli Qi, Ximing Lu, Aaron Tu, Li Erran Li, Yejin Choi · 8 April 2026

  Although RLVR has become an essential component for developing advanced reasoning skills in language models, contemporary studies have documented training plateaus after thousands of optimization steps, i.e., notable decreases in performance gains despite increased computational investment. This lim…

• Cross-fitted Proximal Learning for Model-Based Reinforcement Learning

  Nishanth Venkatesh, Andreas A. Malikopoulos · 8 April 2026

  Model-based reinforcement learning is attractive for sequential decision-making because it explicitly estimates reward and transition models and then supports planning through simulated rollouts. In offline settings with hidden confounding, however, models learned directly from observational data ma…

• Breakthrough the Suboptimal Stable Point in Value-Factorization-Based Multi-Agent Reinforcement Learning

  Lesong Tao, Yifei Wang, Haodong Jing, Jingwen Fu, Miao Kang, Shitao Chen, Nanning Zheng · 8 April 2026

  Value factorization, a popular paradigm in MARL, faces significant theoretical and algorithmic bottlenecks: its tendency to converge to suboptimal solutions remains poorly understood and unsolved. Theoretically, existing analyses fail to explain this due to their primary focus on the optimal case. T…

• Disentangled World Models: Learning to Transfer Semantic Knowledge from Distracting Videos for Reinforcement Learning

  Qi Wang, Zhipeng Zhang, Baao Xie, Xin Jin, Yunbo Wang, Shiyu Wang, Liaomo Zheng, Xiaokang Yang, Wenjun Zeng · 7 April 2026

  Training visual reinforcement learning (RL) in practical scenarios presents a significant challenge, $\textit{i.e.,}$ RL agents suffer from low sample efficiency in environments with variations. While various approaches have attempted to alleviate this issue by disentangled representation learning, …

• Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem

  Turan Orujlu, Christian Gumbsch, Martin V. Butz, Charley M Wu · 7 April 2026

  Most neural models of causality assume static causal graphs, failing to capture the dynamic and sparse nature of physical interactions where causal relationships emerge and dissolve over time. We introduce the Causal Process Framework and its neural implementation, Causal Process Models (CPMs), for …

• Diffusion Policy with Bayesian Expert Selection for Active Multi-Target Tracking

  Haotian Xiang, Qin Lu, Yaakov Bar-Shalom · 7 April 2026

  Active multi-target tracking requires a mobile robot to balance exploration for undetected targets with exploitation of uncertain tracked ones. Diffusion policies have emerged as a powerful approach for capturing diverse behavioral strategies by learning action sequences from expert demonstrations. …

• Apriel-1.5-OpenReasoner: RL Post-Training for General-Purpose and Efficient Reasoning

  Rafael Pardinas, Ehsan Kamalloo, David Vazquez, Alexandre Drouin · 7 April 2026

  Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses si…

• Finite-Time Analysis of Q-Value Iteration for General-Sum Stackelberg Games

  Narim Jeong, Donghwan Lee · 7 April 2026

  Reinforcement learning has been successful both empirically and theoretically in single-agent settings, but extending these results to multi-agent reinforcement learning in general-sum Markov games remains challenging. This paper studies the convergence of Stackelberg Q-value iteration in two-player…

• An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learning

  Xingtu Liu · 7 April 2026

  In this work, we study out-of-distribution (OOD) generalization in meta-reinforcement learning from an information-theoretic perspective. We begin by establishing OOD generalization bounds for meta-supervised learning under two distinct distribution shift scenarios: standard distribution mismatch an…

• Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications

  Zequn Chen, Wesley J. Marrero · 7 April 2026

  Researchers and practitioners are increasingly considering reinforcement learning to optimize decisions in complex domains like robotics and healthcare. To date, these efforts have largely utilized expectation-based learning. However, relying on expectation-focused objectives may be insufficient for…

• Mitigating Value Hallucination in Dyna Planning via Multistep Predecessor Models

  Farzane Aminmansour, Taher Jafferjee, Ehsan Imani, Erin Talvitie, Micheal Bowling, Martha White · 7 April 2026

  Dyna-style reinforcement learning (RL) agents improve sample efficiency over model-free RL agents by updating the value function with simulated experience generated by an environment model. However, it is often difficult to learn accurate models of environment dynamics, and even small errors may res…

• A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning

  Mengqi Li, Lei Zhao, Anthony Man-Cho So, Ruoyu Sun, Xiao Li · 7 April 2026

  Can language models improve their reasoning performance without external rewards, using only their own sampled responses for training? We show that they can. We propose Self-evolving Post-Training (SePT), a simple post-training method that alternates between self-generation and training on self-gene…

• Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards

  Yaoze Guo, Shana Moothedath · 7 April 2026

  Multi-task representation learning (MTRL) is an approach that learns shared latent representations across related tasks, facilitating collaborative learning that improves the overall learning efficiency. This paper studies MTRL for multi-task reinforcement learning (RL), where multiple tasks have th…

• FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control

  Donghu Kim, Youngdo Lee, Minho Park, Kinam Kim, I Made Aswin Nahendra, Takuma Seno, Sehee Min, Daniel Palenicek, Florian Vogt, Danica Kragic, Jan Peters, Jaegul Choo, Hojoon Lee · 7 April 2026

  Reinforcement learning (RL) is a core approach for robot control when expert demonstrations are unavailable. On-policy methods such as Proximal Policy Optimization (PPO) are widely used for their stability, but their reliance on narrowly distributed on-policy data limits accurate policy evaluation i…

• Receding-Horizon Control via Drifting Models

  Daniele Foffano, Alessio Russo, Alexandre Proutiere · 7 April 2026

  We study the problem of trajectory optimization in settings where the system dynamics are unknown and it is not possible to simulate trajectories through a surrogate model. When an offline dataset of trajectories is available, an agent could directly learn a trajectory generator by distribution matc…

• RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets

  Andrew Borthwick, Stephen Ash, Anthony Galczak · 7 April 2026

  2026 has brought an explosion of interest in LLM-guided evolution of agentic artifacts, with systems like GEPA and Autoresearch demonstrating that LLMs can iteratively improve prompts, code, and agent architectures across diverse domains. As adoption accelerates, a central question emerges: given th…