Reinforcement Learning in Robotics

Latest papers

• A Multi-Agent system for Multi-Objective constrained optimization

  Federica Filippini · 19 June 2026

  Many decision-making problems in computing and networking systems can be naturally formulated as cost-minimization problems under performance constraints. In dynamic environments, reinforcement learning (RL) is often used to solve such problems at runtime by embedding both costs and constraint viola…

• Off-Policy Evaluation for Missingness-Aware Policies in MDPs with Rewards Missing Not at Random

  Ziheng Wei, Annie Qu, Rui Miao · 19 June 2026

  In offline Reinforcement Learning, immediate rewards in logged batch data are often unobserved due to sparse or irregular record-keeping, or censored beyond certain reward values. This issue arises in practical settings, including health care and marketing. We investigate off-policy evaluation (OPE)…

• DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning

  Calvin Luo, Chen Sun, Shuran Song · 19 June 2026

  A natural recipe for intelligent robotic decision-making is initializing from pretrained generative control policies, which have summarized offline experience, and adapting them to self-collected online experience. We present DF-ExpEnse, an exploration technique that improves the quality of online e…

• Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots

  Khurram Javed, Joseph Modayil, Gloria Kennickell, Richard S. Sutton, John Carmack · 19 June 2026

  We built a robot called the Robotroller that actuates an Atari CX40+ controller and a device called the Atari Devbox that renders the game frame and the reward signal from the Arcade Learning Environment on a screen. The Robotroller and the Atari Devbox, together with an off-the-shelf camera and a d…

• A Model-Driven Approach for Developing Families of Reinforcement Learning Environments

  Xiaoran Liu, Istvan David · 19 June 2026

  Virtual training environments are software-intensive systems in which reinforcement learning (RL) agents learn, adapt, and demonstrate meaningful behavior. Virtual training environments offer a safe and cost-efficient alternative to training agents in real-world settings. However, to converge, most …

• VOiLA: Vectorized Online Planning with Learned Diffusion Model for POMDP Agents

  Marcus Hoerger, Rishikesh Joshi, Rahul Shome, Ian Manchester, Hanna Kurniawati · 19 June 2026

  Planning under uncertainty is an essential capability for autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for such a capability. Although POMDP-based planning has advanced significantly, its application to real-world problems is often limited…

• Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning

  Hsiao-Ru Pan, Bernhard Sch\"olkopf · 19 June 2026

  Direct Advantage Estimation (DAE) has been shown to improve the sample efficiency of deep reinforcement learning algorithms. However, its reliance on full environment observability limits its applicability in realistic settings, and its requirement to model transition probabilities incurs substantia…

• Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning

  Ismail Geles, Leonard Bauersfeld, Markus Wulfmeier, Davide Scaramuzza · 19 June 2026

  Autonomous systems have achieved superhuman performance in isolation or simulation, yet they remain brittle in shared, dynamic real-world spaces. This failure stems from the dominant single-agent paradigm for physical applications, where other actors are ignored or treated as environmental noise, pr…

• OnDeFog: Online Decision Transformer under Frame Dropping

  Daiki Yotsufuji, Kenta Nishihara, Shoma Shimizu, Kento Uchida, Shinichi Shirakawa · 19 June 2026

  In challenging real-world reinforcement learning applications, communication delays or sensor failures often cause frame dropping, in which the agent cannot receive the dropped states and associated rewards. To address the performance degradation caused by frame dropping, the Decision Transformer un…

• ENPIRE: Agentic Robot Policy Self-Improvement in the Real World

  Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian "Max" Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi "Jim" Fan, Yuke Zhu, Guanya Shi · 19 June 2026

  Achieving dexterous robotic manipulation in the real world heavily relies on human supervision and algorithm engineering, which becomes a central bottleneck in the pursuit of general physical intelligence. Although emerging coding agents can generate code to automate algorithm search, their successe…

• Temporal Self-Imitation Learning

  Yinsen Jia, Boyuan Chen · 19 June 2026

  Long-horizon robot manipulation policies trained with reward shaping can still exploit dense rewards through inefficient interaction, while rare efficient behaviors may be forgotten during training. We argue that temporal efficiency itself provides a powerful and underutilized source of self-supervi…

• Reward as An Agent for Embodied World Models

  Pu Li, Zhigang Lin, Qiang Wu, Yongxuan Lv, Fei Wang, Shan You · 19 June 2026

  While RL has become a promising tool for refining world models, existing methods largely rely on conservative rollouts near the training distribution, limiting exploration, behavioral diversity, and richer dynamic discovery. In this work, we challenge this conservative paradigm. We argue that the co…

• Human-like autonomy emerges from self-play and a pinch of human data

  Daphne Cornelisse, Julian Hunt, Zixu Zhang, Wa\"el Doulazmi, Kevin Joseph, Jaime Fern\'andez Fisac, Eugene Vinitsky · 19 June 2026

  Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play…

• Playful Agentic Robot Learning

  Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, Trevor Darrell · 19 June 2026

  Current agentic robot systems can write executable Code-as-Policy programs, observe feedback, and revise behavior across multiple attempts, but they remain largely task-driven: reusable skills are acquired only after explicit instructions. We study Playful Agentic Robot Learning, where an embodied c…

• CRAX: Fast Safe Reinforcement Learning Benchmarking

  Tristan Tomilin, Mourad Boustani, Mickey Beurskens, Thiago D. Sim\~ao · 19 June 2026

  Safety is a core concern for deploying reinforcement learning (RL) agents in real-world domains such as robotics and autonomous driving. While benchmarks have been central to progress in RL, existing safety benchmarks with high-fidelity 3D physics remain computationally slow, limiting large-scale ex…

• Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution

  Jannik H\"osch, Alessandro Sestini, Florian Fuchs, Amir Baghi, Joakim Bergdahl, Konrad Tollmar, Jean-Philippe Barrette-LaPierre, Linus Gissl\'en · 19 June 2026

  Reinforcement learning (RL) has achieved strong performance in sequential decision-making, yet scaling to complex multi-agent environments remains challenging due to sparse rewards, large state-action spaces, and the difficulty of learning coordinated strategies. We propose a hierarchical architectu…

• MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

  Tristan Tomilin, Luka van den Boogaard, Samuel Garcin, Constantin Ruhdorfer, Bram Grooten, Fabrice Kusters, Yali Du, Andreas Bulling, Mykola Pechenizkiy, Meng Fang · 19 June 2026

  Benchmarks play a central role in reinforcement learning (RL) research, yet their computational constraints often shape what is studied. Despite the motivation of lifelong learning, most continual RL papers consider only 3-10 sequential tasks, as CPU-bound environments make longer sequences impracti…

• Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methods

  Jeong Woon Lee, Kyoleen Kwak, Daeho Kim, Hyoseok Hwang · 19 June 2026

  Policies learned via continuous actor-critic methods often exhibit erratic, high-frequency oscillations, making them unsuitable for physical deployment. Current approaches attempt to enforce smoothness by directly regularizing the policy's output. We argue that this approach treats the symptom rathe…

• When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?

  Xuanfei Ren, Tengyang Xie · 18 June 2026

  Offline reinforcement learning is typically analyzed under process-level reward supervision, yet many sequential decision datasets record only trajectory-level outcomes. We develop a statistical theory for offline policy optimization from such outcome-level supervision. We first study the canoni…

• UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

  Mohamed Nabail, Leo Cheng, Jingmin Wang, Nicholas Rhinehart · 18 June 2026

  Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design. However, existing methods typically rely on passive data collection and suffer from poor sample efficiency, especially during the early stages of …

• Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards

  Yingyu Shan, Yuhang Guo, Zihao Cheng, Zeming Liu, Xiangrong Zhu, Xinyi Wang, Jiashu Yao, Wei Lin, Hongru Wang, Heyan Huang · 18 June 2026

  Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps. Existing toke…

• Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents

  Caleb Chang, Davin Win Kyi, Natasha Jaques, Karen Leung · 18 June 2026

  Humans often acquire new skills by observing others, since observed behaviors implicitly reveal how to act in an environment. However, observations drawn from a heterogeneous population introduce conflicting behavioral signals, making it difficult to determine which behaviors are worth imitating. We…

• Maturing Markov Decision Processes: Decision Making under Increasing Information and Shrinking Action Sets

  Jiaxi Liu, Aiping Yang, Yuhang Yang, Shuqi Zhang, Zewei Dong, Jiangming Yang, Xuebin Chen · 18 June 2026

  Sequential decision problems often exhibit an asymmetric evolution of information and decision flexibility: as a decision cycle unfolds, the agent receives richer information while feasible actions expire due to operational cutoffs, commitments, or resource constraints. Standard MDP formulations typ…

• LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents

  Haoyang Fang, Wei Zhu, Boran Han, Alex Zhang, Zhenyu Pan, Shuo Yang, Shuai Zhang, Jiading Gai, Peng Tang, Cuixiong Hu, Xuan Zhu, Huzefa Rangwala, George Karypis, Bernie Wang · 18 June 2026

  RL post-training strategies are dataset-dependent and reveal a recurring empirical pattern: capacity parameters accumulate monotonically across stages, while regularization parameters predominantly oscillate in response to shifting training dynamics. This distinction matters because fixed schedules …

• Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontier

  Lorenz Wolf, Connor Watts, Roger Creus Castanyer, Geoffrey Bradway, Maxwill Lin, Augustine N. Mavor-Parker, Matthew Daborn-Sargent · 18 June 2026

  The limiting resource for training agents via reinforcement learning (RL) is increasingly frontier task supply: valid, solvable tasks just difficult enough to train the current model. As reasoning and agentic models improve, fixed task distributions saturate, while naive synthetic generation yields …