Reinforcement Learning in Robotics

Últimos artículos

• When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents

  Junzhuo Liu, Weiwei Li, Jun Ling, Peng Wang · 7 de agosto de 2026

  Privileged on-policy distillation provides dense supervision for multi-turn agents by allowing a synchronized teacher to re-score the student's response at every turn with access to training-only references, such as successful trajectories. In interactive environments, however, the student's precedi…

• AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning

  Zi-Han Wang, Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Jie Wu, Zhengzhou Cai, Yueqing Sun, Ziang Ye, Linji Hao, Qi Gu, Xunliang Cai, Yongliang Shen, Yujiu Yang · 7 de agosto de 2026

  Reinforcement learning (RL) with verifiable rewards constructs trajectory-level advantage estimates, yet it often fails to credit the few pivotal decisions that determine outcomes in long-horizon, multi-turn agentic tasks. Recent work introduces privileged self-distillation for credit assignment, pr…

• ProDVI: Programmatic Dynamics Priors for Value Network Initialization

  Xinwei Liu, Junyuan Liang, Jianting Zhang, Wuhui Chen · 7 de agosto de 2026

  Deep Reinforcement Learning (RL) is notoriously sample inefficient. One contributing factor is that RL agents are typically initialized from scratch, forcing them to acquire task-relevant knowledge through online interaction. Existing approaches obtain informative initializations through pre-collect…

• RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction

  Chenglong Wang, Ziming Zhu, Yifu Huo, Bei Li, Qiaozhi He, Yan Ding, Xiaoyang Hao, Yuxin Gao, Tianhua Zhou, Xiaojia Chang, Tongran Liu, Jingbo Zhu · 7 de agosto de 2026

  Recent advances in reward modeling show a paradigm shift from discriminative reward models to generative reward models. However, despite their strong capabilities in response ranking, generative reward models have not realized their potential in reinforcement learning (RL). Our analysis reveals that…

• Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback

  Zeqiang Zhang, Fabian Wurzberger, Gerrit Schmid, Sebastian Gottwald, Daniel A. Braun · 7 de agosto de 2026

  Learning from reward functions and imitation learning of demonstrations are the two principal approaches for training autonomous systems that interact with an environment through action and observation. Both, however, require human specification for each behaviour to be acquired, a problem for long-…

• Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control

  Xinwei Liu, Junyuan Liang, Jianting Zhang, Wuhui Chen · 7 de agosto de 2026

  Sample-efficient policy learning from pixels is a long-standing challenge in reinforcement learning (RL). Recent dynamics-based representation learning methods have significantly improved the sample efficiency of model-free visual RL by learning dynamics-aware representations through auxiliary predi…

• Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning

  Yifan Wu, Junjie Lei, Wenjie Huang · 6 de agosto de 2026

  Risk-aware Q-learning (RaQL) provides a model-free, two-timescale estimator for dynamic risk objectives, but its finite-budget behavior remains fragile: fixed inner-loop hyperparameters can produce unstable value estimates, persistent Bellman residuals, and inefficient sample reuse. This paper propo…

• ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning

  R. Blake Lawlor, Daniel S. Brown · 6 de agosto de 2026

  Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficiency, but still fail when the environment shifts. This paper introduces A…

• Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning

  Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing Kuang, Yanfang Ye, Wei Niu · 6 de agosto de 2026

  Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models. However, most existing methods allocate the same number of rollouts to every task and trajectory state, even though some rollouts provide much more useful learning signals than othe…

• Curiosity-Diffuser: Curiosity Guide Diffusion Models for Reliability

  Zihao Liu, Xing Liu, Yuhang Dong, Haitao Chang, Zhengxiong Liu, Panfeng Huang · 6 de agosto de 2026

  One of the bottlenecks in robotic intelligence is the instability of neural network models. This leads to risks when applying intelligence in the physical world. Specifically, imitation policy based on neural network may generate hallucinations, leading to inaccurate behaviors that impact the safety…

• State2State: Environment-Derived Mid-Training for LLM Agents

  Xuanyu Lei, Yiqi Zhu, Chenliang Li, Kaiming Liu, Peng Li, Ming Yan, Jieping Ye, Ya-Qin Zhang, Yang Liu · 6 de agosto de 2026

  Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specified tasks and supervision signals, limiting the scalability an…

• Not Every Divergence Should Be Suppressed: Counterfactual Recoverability in On-Policy Distillation

  De Jiang, Zhengyang Zhang, Kehong Yuan, Shaohua Ma · 6 de agosto de 2026

  On-policy distillation (OPD) supervises student-visited trajectories, yet divergence-based rules cannot determine whether an erroneous prefix remains correctable. We formulate this decision as counterfactual recoverability and replay each error state through budget-matched teacher-continuation and r…

• WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models

  Bohai Gu, Yueyang Yuan, Taiyi Wu, Dazhao Du, Jian Liu, Xiaoyi Pang, Jie Zhang, Xiaocheng Lu, Haobin Zhong, Xiaotong Zhao, Alan Zhao, Song Guo · 6 de agosto de 2026

  Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors. Post-training methods such as reinforcement learning (RL) can improve these models, but they hit a verification bottleneck: for arbitrary action sequences, no ground-truth…

• Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning

  Jai Malegaonkar, Rohan Patil, Henrik I. Christensen · 6 de agosto de 2026

  In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies. Exploration bonuses and memory architectures are traditionally evaluated in isolation, leaving their interac…

• Learning When to Stop: Prefix-Optimal Dynamic Diffusion Policies for Continuous Control

  Rohit Kumar Salla, Manoj Saravanan, Simon Stepputtis · 6 de agosto de 2026

  Diffusion policies are a powerful policy class for continuous control, but their iterative denoising process creates a substantial computational bottleneck. Reducing this cost requires adapting the number of denoising steps to the difficulty of each action while preserving task performance. We intro…

• Toward Integrating Adaptive Experience Replay and Online Uncertainty Estimation in Safe Actor-Critic Optimal Control

  Mahshad Rastegarmoghaddam, Davoud Nikkhouy, Shima Samadzadeh · 6 de agosto de 2026

  Safe actor-critic control often treats barrier filtering, uncertainty estimation, and experience replay as separate modules, even though each changes the data used for learning and control. We develop an integrated architecture in which the uncertainty estimate updates the obstacle geometry used by …

• When Correct Solutions Repeat: Rarity-Aware Credit Redistribution for GRPO

  Zhe Cao, Miaowen Wen, Fangjiong Chen · 5 de agosto de 2026

  Reinforcement learning with verifiable rewards (RLVR) com- monly optimizes each correct completion as an independent learning signal. In GRPO, this completion-level uniformity creates structure-level skew: recurring correct solution forms accumulate positive coefficient mass in proportion to how oft…

• Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning

  Pyrros Koussios, Chenhao Li, Xin Chen, Andreas Krause · 5 de agosto de 2026

  Designing effective reward functions remains a major bottleneck in Reinforcement Learning (RL). Recent work uses large foundation Vision-Language Models (VLMs) as reward models, computing text-observation similarity to bypass manual reward engineering. Although promising, these rewards are often noi…

• SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation

  Wen Wang, Jiahua Bao, Tu Yongsiqi, Yihao Liu, Haotian Zhou, Haoxuan Ma, Mengyu Zhou, Wenkui Fan, Junwei He, Xiaoxi Jiang, Guanjun Jiang · 5 de agosto de 2026

  We aim to improve model performance in multi-reward reinforcement learning training process. Existing Group reward-Decoupled Normalization Policy Optimization (GDPO) has mitigated the issue of reward signals masking one another during direct scalarization by normalizing each reward dimension separat…

• SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling

  Evan Assmus, Qining Zhang, Lei Ying · 5 de agosto de 2026

  Preference-based reinforcement learning (PbRL) for general stochastic MDPs often requires training a reward model. Existing reward-model-free methods are either restricted to bandits or deterministic MDPs, such as DPO or P3O, or use zeroth-order, gradient-free optimization, which in general exhibits…

• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL

  Yi Yang, Zhennan Chen, Mingfeng Lv, Hanlei Li, Zhengsen Ruan, Lvqing Yang · 5 de agosto de 2026

  Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping. Existing regularization and local-generalization methods control either the admissible OOD region or the influence of gener…

• Revisiting TD Target Aggregation under Uncertainty in Q-Learning

  Lipeng Zu, Xiaonan Zhang · 5 de agosto de 2026

  Deep Q-Networks (DQNs) learn value functions through bootstrapped temporal-difference updates, where future returns are approximated using a greedy maximization over next-state action values. While effective, this aggregation rule is inherently sensitive to estimation noise: when Q-values are uncert…

• Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Exploration

  Amir Baghi, Jens Sj\"olund, Joakim Bergdahl, Linus Gissl\'en, Alessandro Sestini · 5 de agosto de 2026

  Multi-agent reinforcement learning has shown promise in learning cooperative behaviors in team-based environments. However, such methods often demand extensive training time, which inhibits their application for game-AI in standard game development. For instance, the state-of-the-art method TiZero t…

• Robust General Utility for Reinforcement Learning

  Zixuan Liu, Fangzheng Wu, Brian Summa, Zizhan Zheng · 5 de agosto de 2026

  Reinforcement learning (RL) with general utility extends classic RL by optimizing an arbitrary utility functional of the policy-induced occupancy measure, thereby enabling a broader range of applications. However, previous work on general utility RL typically assumes the evaluation utility is fixed …

• Verifier-Induced Support Reshaping in On-Policy Optimization

  Shaohang Wei, Zikun Su, Feifan Song, Wen Luo, Wei Li, Guangyue Peng, Houfeng Wang · 4 de agosto de 2026

  We show that on-policy reinforcement learning with verifiable rewards (RLVR) can improve the current objective while making successful behaviors for later objectives too rare to sample and reinforce. We call this verifier-induced support reshaping and define effective rewardable support as successfu…