Reinforcement Learning in Robotics

Latest papers

• ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors

  Zifan Xu, Ran Gong, Maria Vittoria Minniti, Ahmet Salih Gundogdu, Eric Rosen, Kausik Sivakumar, Riedana Yan, Zixing Wang, Di Deng, Peter Stone, Xiaohan Zhang, Karl Schmeckpeper · 18 March 2026

  Learning generalizable and robust behavior cloning policies requires large volumes of high-quality robotics data. While human demonstrations (e.g., through teleoperation) serve as the standard source for expert behaviors, acquiring such data at scale in the real world is prohibitively expensive. Thi…

• Anticipatory Planning for Multimodal AI Agents

  Yongyuan Liang, Shijie Zhou, Yu Gu, Hao Tan, Gang Wu, Franck Dernoncourt, Jihyung Kil, Ryan A. Rossi, Ruiyi Zhang · 18 March 2026

  Recent advances in multimodal agents have improved computer-use interaction and tool-usage, yet most existing systems remain reactive, optimizing actions in isolation without reasoning about future states or long-term goals. This limits planning coherence and prevents agents from reliably solving hi…

• You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector

  Omkar Patil, Ondrej Biza, Thomas Weng, Karl Schmeckpeper, Wil Thomason, Xiaohan Zhang, Robin Walters, Nakul Gopalan, Sebastian Castro, Eric Rosen · 18 March 2026

  What happens when a pretrained generative robot policy is provided a constant initial noise as input, rather than repeatedly sampling it from a Gaussian? We demonstrate that the performance of a pretrained, frozen diffusion or flow matching policy can be improved with respect to a downstream reward …

• CABTO: Context-Aware Behavior Tree Grounding for Robot Manipulation

  Yishuai Cai, Xinglin Chen, Yunxin Mao, Kun Hu, Minglong Li, Yaodong Yang, Yuanpei Chen · 18 March 2026

  Behavior Trees (BTs) offer a powerful paradigm for designing modular and reactive robot controllers. BT planning, an emerging field, provides theoretical guarantees for the automated generation of reliable BTs. However, BT planning typically assumes that a well-designed BT system is already grounded…

• Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models

  Yanru Wu, Weiduo Yuan, Ang Qi, Vitor Guizilini, Jiageng Mao, Yue Wang · 18 March 2026

  Reinforcement Learning (RL) has shown great potential in refining robotic manipulation policies, yet its efficacy remains strongly bottlenecked by the difficulty of designing generalizable reward functions. In this paper, we propose a framework for online policy refinement by adapting foundation VLM…

• Regularized Latent Dynamics Prediction is a Strong Baseline For Behavioral Foundation Models

  Pranaya Jajoo, Harshit Sikchi, Siddhant Agarwal, Amy Zhang, Scott Niekum, Martha White · 18 March 2026

  Behavioral Foundation Models (BFMs) produce agents with the capability to adapt to any unknown reward or task. These methods, however, are only able to produce near-optimal policies for the reward functions that are in the span of some pre-existing state features, making the choice of state features…

• ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learning

  Yu Li, Rui Miao, Zhengling Qi, Tian Lan · 18 March 2026

  The dominant paradigm for improving mathematical reasoning in language models relies on Reinforcement Learning with verifiable rewards. Yet existing methods treat each problem instance in isolation without leveraging the reusable strategies that emerge and accumulate during training. To this end, we…

• Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

  Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil · 18 March 2026

  Simulation-to-real transfer remains a central challenge in robotics, as mismatches between simulated and real-world dynamics often lead to failures. While reinforcement learning offers a principled mechanism for adaptation, existing sim-to-real finetuning methods struggle with exploration and long-h…

• When a Robot is More Capable than a Human: Learning from Constrained Demonstrators

  Xinhu Li, Ayush Jain, Zhaojing Yang, Yigit Korkmaz, Erdem B{\i}y{\i}k · 18 March 2026

  Learning from demonstrations enables experts to teach robots complex tasks using interfaces such as kinesthetic teaching, joystick control, and sim-to-real transfer. However, these interfaces often constrain the expert's ability to demonstrate optimal behavior due to indirect control, setup restrict…

• Counteractive RL: Rethinking Core Principles for Efficient and Scalable Deep Reinforcement Learning

  Ezgi Korkmaz · 18 March 2026

  Following the pivotal success of learning strategies to win at tasks, solely by interacting with an environment without any supervision, agents have gained the ability to make sequential decisions in complex MDPs. Yet, reinforcement learning policies face exponentially growing state spaces in high d…

• On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting

  Wenhao Zhang, Yuexiang Xie, Yuchang Sun, Yanxi Chen, Guoyin Wang, Yaliang Li, Bolin Ding, Jingren Zhou · 18 March 2026

  Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are two prominent post-training paradigms for refining the capabilities and aligning the behavior of Large Language Models (LLMs). Existing approaches that integrate SFT and RL often face the risk of disrupting established response pattern…

• From $\boldsymbol{\log\pi}$ to $\boldsymbol{\pi}$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight

  Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Chaowen Hu, Cong Qin, Zekai Shao, Binbin Zheng, Lu Pan, Ke Zeng · 17 March 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed a leap in Large Language Model (LLM) reasoning, yet its optimization dynamics remain fragile. Standard algorithms like GRPO enforce stability via ``hard clipping'', which inadvertently stifles exploration by discarding gradients of …

• Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies

  Mumuksh Tayal, Manan Tayal, Ravi Prakash · 17 March 2026

  Offline safe reinforcement learning (RL) seeks reward-maximizing policies from static datasets under strict safety constraints. Existing methods often rely on soft expected-cost objectives or iterative generative inference, which can be insufficient for safety-critical real-time control. We propose …

• Sample-Efficient Hypergradient Estimation for Decentralized Bi-Level Reinforcement Learning

  Mikoto Kudo, Takumi Tanabe, Akifumi Wachi, Youhei Akimoto · 17 March 2026

  Many strategic decision-making problems, such as environment design for warehouse robots, can be naturally formulated as bi-level reinforcement learning (RL), where a leader agent optimizes its objective while a follower solves a Markov decision process (MDP) conditioned on the leader's decisions. I…

• Chunk-Guided Q-Learning

  Gwanwoo Song, Kwanyoung Park, Youngwoon Lee · 17 March 2026

  In offline reinforcement learning (RL), single-step temporal-difference (TD) learning can suffer from bootstrapping error accumulation over long horizons. Action-chunked TD methods mitigate this by backing up over multiple steps, but can introduce suboptimality by restricting the policy class to ope…

• Pixel-level Scene Understanding in One Token: Visual States Need What-is-Where Composition

  Seokmin Lee, Yunghee Lee, Byeonghyun Pak, Byeongju Woo · 17 March 2026

  For robotic agents operating in dynamic environments, learning visual state representations from streaming video observations is essential for sequential decision making. Recent self-supervised learning methods have shown strong transferability across vision tasks, but they do not explicitly address…

• GoldenStart: Q-Guided Priors and Entropy Control for Distilling Flow Policies

  He Zhang, Ying Sun, Hui Xiong · 17 March 2026

  Flow-matching policies hold great promise for reinforcement learning (RL) by capturing complex, multi-modal action distributions. However, their practical application is often hindered by prohibitive inference latency and ineffective online exploration. Although recent works have employed one-step d…

• Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization

  Yanning Dai, Yuhui Wang, Dylan R. Ashley, J\"urgen Schmidhuber · 17 March 2026

  Morphology-control co-design concerns the coupled optimization of an agent's body structure and control policy. This problem exhibits a bi-level structure, where the control dynamically adapts to the morphology to maximize performance. Existing methods typically neglect the control's adaptation dyna…

• Physics-Informed Policy Optimization via Analytic Dynamics Regularization

  Namai Chandra, Liu Mohan, Zhihao Gu, Lin Wang · 17 March 2026

  Reinforcement learning (RL) has achieved strong performance in robotic control; however, state-of-the-art policy learning methods, such as actor-critic methods, still suffer from high sample complexity and often produce physically inconsistent actions. This limitation stems from neural policies impl…

• Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner

  Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Yufeng Zhong, Lin Ma · 17 March 2026

  Chart reasoning presents unique challenges due to its inherent complexity -- requiring precise numerical comprehension, multi-level visual understanding, and logical inference across interconnected data elements. Existing vision-language models often struggle with such reasoning tasks, particularly …

• MA-VLCM: A Vision Language Critic Model for Value Estimation of Policies in Multi-Agent Team Settings

  Shahil Shaik, Aditya Parameshwaran, Anshul Nayak, Jonathon M. Smereka, Yue Wang · 17 March 2026

  Multi-agent reinforcement learning (MARL) commonly relies on a centralized critic to estimate the value function. However, learning such a critic from scratch is highly sample-inefficient and often lacks generalization across environments. At the same time, large vision-language-action models (VLAs)…

• Learning to Order: Task Sequencing as In-Context Optimization

  Jan Kobiolka, Christian Frey, Arlind Kadra, Gresa Shala, Josif Grabocka · 17 March 2026

  Task sequencing (TS) is one of the core open problems in Deep Learning, arising in a plethora of real-world domains, from robotic assembly lines to autonomous driving. Unfortunately, prior work has not convincingly demonstrated the generalization ability of meta-learned TS methods to solve new TS pr…

• Implicit Maximum Likelihood Estimation for Real-time Generative Model Predictive Control

  Grayson Lee, Minh Bui, Shuzi Zhou, Yankai Li, Mo Chen, Ke Li · 17 March 2026

  Diffusion-based models have recently shown strong performance in trajectory planning, as they are capable of capturing diverse, multimodal distributions of complex behaviors. A key limitation of these models is their slow inference speed, which results from the iterative denoising process. This make…

• Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling

  Shuyang Jiang, Yusheng Liao, Ya Zhang, Yanfeng Wang, Yu Wang · 17 March 2026

  While large reasoning models trained with critic-free reinforcement learning and verifiable rewards (RLVR) represent the state-of-the-art, their practical utility is hampered by ``overthinking'', a critical issue where models generate excessively long reasoning paths without any performance benefit.…

• Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State

  Ziheng Cheng, Xin Guo, Yufei Zhang · 17 March 2026

  The theory of continuous-time reinforcement learning (RL) has progressed rapidly in recent years. While the ultimate objective of RL is typically to learn deterministic control policies, most existing continuous-time RL methods rely on stochastic policies. Such approaches often require sampling acti…