Reinforcement Learning in Robotics

Latest papers

• DriftWorld: Fast World Modeling through Drifting

  Susie Lu, Haonan Chen, Weirui Ye, Yilun Du · 17 July 2026

  Predictive world models enable robots to plan by imagining the outcomes of their actions, but their value for control hinges on generating many rollouts quickly. This creates a bottleneck for diffusion-based world models: multistep sampling makes each rollout expensive, limiting large-scale action s…

• PAC Learning in Turn-Based Stochastic Games with Reachability Objectives: A Decentralized Private Approach via Expected Conditional Distance

  Ali Asadi, Krishnendu Chatterjee, Pavol Kebis · 17 July 2026

  Reachability is the most fundamental logical objective, yet it is notoriously difficult to learn in reinforcement learning settings: even for Markov decision processes, PAC learning of reachability is impossible without additional assumptions. This difficulty also holds in turn-based stochastic game…

• Lyapunov Guidance: A Unified Framework for Stabilizing Generative Flows

  Jingdong Zhang, Xinze Li, Yize Jiang, Luan Yang, Minkai Xu, Junhong Liu · 17 July 2026

  Flow matching has emerged as an effective framework for learning complex data distributions, but adapting pretrained flow models to new tasks often requires computationally expensive retraining. Post-training guidance provides a more efficient alternative, but existing methods are largely heuristic …

• Concept-Guided Spatial Regularization for World Models in Atari Pong

  Yukuan Lu, Zaishuo Xia, Weyl Lu, Yubei Chen · 17 July 2026

  World models are usually evaluated as components of model-based reinforcement learning (MBRL) systems, while the world models themselves are rarely studied in isolation. We examine five representative visual world-model agents in Atari Pong: DreamerV3, DIAMOND, TWISTER, Simulus, and STORM. After r…

• DNQ: Deep Nash Q-Network for Partially Observable n-Player Games

  Qintong Xie, Edward Koh, Xavier Cadet, Peter Chin · 17 July 2026

  Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interaction, as in auctions, resource allocation, and security competition. We study multi-turn simultaneous bidding as a controlled testbed for such…

• MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generators

  Yushi Huang, Xiangxin Zhou, Jun Zhang, Liefeng Bo, Tianyu Pang · 17 July 2026

  MeanFlow generators achieve fast few-step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task-specific objectives. In p…

• Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization

  Weiwen Xu, Jia Liu, Hou Pong Chan, Long Li, Deng Cai, Min Chen, Hao Zhang · 17 July 2026

  Reinforcement learning with verifiable rewards (RLVR) commonly uses entropy for advantage shaping. However, entropy cannot distinguish useful uncertainty from detrimental confusion, limiting its effectiveness as a correctness signal. We propose Contrastive Policy Optimization (CPO), which uses token…

• A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models

  Zikun Zhang, Jiayuan Sheng, David D. Yao, Wenpin Tang · 17 July 2026

  We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chain (CTMC). We consider policy optimization problems and derive…

• On-Policy Delta Distillation

  Byeongho Heo, Jaehui Hwang, Sangdoo Yun, Dongyoon Han · 17 July 2026

  On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervision from a teacher model. Although on-policy distillation has been studied and applied across various settings, its fundam…

• Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control

  Jihoon Hong, Julian Skifstad, Qiyue Dai, Alice Chan, Glen Chou · 17 July 2026

  World Action Models (WAMs) enable semantically- and physically-informed control but are brittle under distribution shift. In this work, we use mechanistic interpretability to study how robustness-relevant perturbations are represented in WAM activation space. Comparing activations across successful …

• Where Should RL Post-Training Compute Go? Model Size, Search, Learning, and Feedback

  Patrick Wilhelm, Odej Kao · 16 July 2026

  Reinforcement Learning (RL) post-training is increasingly used to adapt foundation models for reasoning, planning, and feedback-driven robot-learning pipelines, but constrained post-training resources are often summarized by a single total FLOP budget. We study the fixed-budget decision problem behi…

• Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations

  Rui Wang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Wenhao Yu, Kam-Fai Wong · 16 July 2026

  On-policy distillation (OPD) has become a key paradigm in LLM post-training, yet its training dynamics remain poorly understood. We present a systematic study examining the role, pathologies, and regulations of OPD. We first clarify the role of OPD as an exploration catalyst: it steers the student t…

• Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models

  Ilias Kazantzidis, Timothy J. Norman, Yali Du, Christopher T. Freeman · 16 July 2026

  We address the problem of safely training an agent policy and deploying a good and safe policy, in settings where the environment dynamics are unknown and no suitable reward function is available. In the context of safety-critical environments, we consider traditional reinforcement learning impracti…

• Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processes

  Jonas Ehrhardt, Ren\'e Heesch, Oliver Niggemann · 16 July 2026

  In this paper, we study Reinforcement Learning in Parametrized Action Markov Decision Processes (PAMDP), where each decision consists of a symbolic action and numerical parameters. In such settings Reinforcement Learning algorithms typically determine parameters with one-shot estimators, which makes…

• Deconstructing Actor-Critic: A Large-scale Empirical Study of Design Components for Practitioners

  Haseeb Shah, Lingwei Zhu, Adam White, Martha White · 16 July 2026

  Reinforcement learning is increasingly being considered for controlling real-world systems, from fusion plasma and autonomous vehicles to drug discovery and drinking water treatment, where reliability is essential and tuning budgets are limited. Actor-critic algorithms share a set of design decision…

• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

  Leitian Tao, Baolin Peng, Wenlin Yao, Tao Ge, Hao Cheng, Mike Hang Wang, Jianfeng Gao, Sharon Li · 16 July 2026

  Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training. Outcome rewards provide reliable supervision for short-horizon reasoning, but become sparse and high-variance …

• Distributionally Robust and Safe Imitation Learning

  Ahmed Aboudonia, Naira Hovakimyan · 16 July 2026

  Imitation learning (IL) has achieved remarkable success in complex decision-making tasks. However, its performance is highly sensitive to distribution shifts, which can pose significant safety risks. We propose a distributionally robust and safe IL framework that explicitly addresses both policy-ind…

• SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy

  Yassine Chemingui, Chenhua Fan, Honghao Wei, Janardhan Rao Doppa · 16 July 2026

  Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces SteinGate, a boundary-aware distributional safety certificate that replaces frag…

• Factorized Spectral Representations for Reinforcement Learning

  Junyi Wu, Dan Li · 16 July 2026

  Learning a compact model of the world from interaction data is central to sample-efficient deep reinforcement learning. Spectral representation methods have become the leading paradigm for representation learning in continuous control by taking a matrix view of the transition kernel, with state-acti…

• Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

  Wenxiao Wang, Priyatham Kattakinda, Soheil Feizi · 16 July 2026

  Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed agents, where optimization is applied re…

• DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention

  Xing Lei, Wenyan Yang, Xuetao Zhang, Donglin Wang · 16 July 2026

  Goal-conditioned reinforcement learning hinges on how the goal is encoded. Contrastive, metric, temporal-distance, and information-theoretic encoders differ in objective. They still share one trait. None of them sees the current state. Such a state-independent embedding cannot mark which part of the…

• Diversity-Enriched Option-Critic

  Anand Kamat, Doina Precup · 15 July 2026

  Temporal abstraction allows reinforcement learning agents to represent knowledge and develop strategies over different temporal scales. The option-critic framework has been demonstrated to learn temporally extended actions, represented as options, end-to-end in a model-free setting. However, feasibi…

• Mind the Gap: Promises and Pitfalls of Hierarchical Planning in LeWorldModel

  Niccol\`o Caselli, Salvatore Lo Sardo, Francesco Massafra, Ippokratis Pantelidis, Samuele Punzo, Sathya Kamesh Bhethanabhotla · 15 July 2026

  We investigate whether temporal hierarchy can improve LeWorldModel on long-horizon goal-conditioned control. We introduce Hi-LeWM, an extension that freezes the pretrained low-level LeWM and adds high-level planning over latent subgoals. We evaluate Hi-LeWM on PushT and Cube across increasing goal o…

• Environment Parameter Gradient Theorem for Policy-Environment Co-Design in Reinforcement Learning

  Amber Srivastava · 15 July 2026

  Reinforcement learning (RL) is traditionally concerned with learning a control policy for a fixed environment. In many engineering systems, however, the environment itself is alterable: physical or operational parameters can be tuned to shape the transition dynamics and costs experienced by the agen…

• Directional Constraints for Efficient Exploration in Safe Reinforcement Learning

  Paolo Magliano, Puze Liu, Jan Peters, Davide Tateo, Raffaello Camoriano · 15 July 2026

  Reinforcement Learning has revolutionized the landscape of robotic research, allowing robust learning of complex robotic skills in simulation. However, real-world deployment in open-ended environments requires strong safety guarantees to prevent dangerous or harmful behaviors. Safe Reinforcement Lea…