Reinforcement Learning in Robotics

Últimos artículos

• Mathematical methods of reinforcement learning

  Denis Belomestny, Alexander Gasnikov, Egor Gladin, Alexey Naumov, Artemy Rubtsov, Yuri Sapronov, Daniil Tiapkin, Nikita Yudin · 9 de julio de 2026

  Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory. This survey organizes the mathematical structures that underpin the design and analysis of modern algorithms in RL. We begin from Markov decision processes (MDPs) and the Bellman operat…

• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning

  Zhenyu Hou, Yujiang Li, Jie Tang, Yuxiao Dong · 9 de julio de 2026

  Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs). Previous RL pipelines for LLMs were mostly synchronous and batch-interleaved, which is inefficient for long-horizon agentic tasks. Recently, asynchronous RL has emerged as a more efficient …

• Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

  Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay · 9 de julio de 2026

  Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences. However, applying RLHF to diffusion models remains highly feedback inefficient, as existing approaches typically require large amounts of human or reward model …

• Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization

  Shota Takashiro, Masanori Koyama, Takeru Miyato, Yusuke Iwasawa, Yutaka Matsuo, Kohei Hayashi · 9 de julio de 2026

  We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory. Our focus is on a persistent fast slow recurrent formulation in which a latent state is maintained across observations …

• RLVP: Penalize the Path, Reward the Outcome

  Bojie Li, Noah Shi · 9 de julio de 2026

  Agents acting on our behalf in the real world (e.g. placing phone calls) must learn online from costly, often irreversible interactions rather than cheap simulator steps. Two things follow. First, deployability depends on the path, not only the outcome. An agent must respect outcome-neutral constrai…

• Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies

  Dennis Gross, Quentin Mazouni, Helge Spieker, Arnaud Gotlieb · 9 de julio de 2026

  Reinforcement learning (RL) policies can be unsafe and vulnerable to attacks. Ensuring their reliability is often a pain point as existing automated testing methods target only selected environments, testing scenarios, and RL algorithms. To address this, we propose a comprehensive framework for test…

• Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning

  Zetian Hu, Shunyu Liu, Junjie Zhang, Yongcheng Jing, Ting-En Lin, Yongbin Li, Dacheng Tao · 9 de julio de 2026

  Recent breakthroughs of Reinforcement Learning (RL) have highlighted its potential for complex agentic Large Language Model (LLM) tasks. However, existing efforts largely focus on single-task settings, whereas real-world deployment necessitates a generalist agent capable of solving multiple tasks si…

• Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning

  Vladislav Beliaev · 9 de julio de 2026

  Reinforcement learning from verifiable rewards (e.g. GRPO) is the engine behind today's reasoning models, yet it grades only the final answer. On hard problems this trains models to write more rather than to think better, since the trace itself is never graded and no label for good thinking exists. …

• Safe Reinforcement Learning using Ideas from Model Predictive Control

  Georg Sch\"afer, Jakob Rehrl, Stefan Huber, Simon Hirlaender · 9 de julio de 2026

  Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world ph…

• Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems

  Vladislav Beliaev · 9 de julio de 2026

  Group Relative Policy Optimization (GRPO) stalls on a model's hardest problems: when no rollout in a group succeeds, the group-relative advantages vanish and the problem contributes no gradient, wasting the frontier examples we most want to learn from. Prepending a correct prefix of a reference solu…

• Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?

  Spandan Garg, Vikram Nitin, Yufan Huang · 9 de julio de 2026

  Modern coding agents increasingly delegate specialized subtasks to subagents, which are smaller, focused agentic loops that handle narrow responsibilities like search, debugging or terminal execution. This architectural pattern keeps the main agent's context window clean by isolating verbose outputs…

• KAT-Coder-V2.5 Technical Report

  Bo Huang, Fengxiang Li, Hao Xu, Haoyang Huang, Hongyi Fu, Jinhua Hao, Kun Yuan, Minglei Zhang, Pengcheng Xu, Shiyang Liu, Wenhao Zhuang, Yuze Shi, Zongxian Feng, Chao Wang, Cheng He, Chongling Rao, Deyu Cao, Fan Yang, Gang Xiong, Haochen Liu, Jiabao Li, Jian Liang, Jinghui Jia, Jingwen Chang, Jun Du, Junyu Shi, Min Li, Mingqi Wu, Qiang Gao, Shangpeng Yan, Shaotong Qi, Shu Xu, Shuo Zhou, Tiankuo Xu, Tong Zheng, Weilun Zhao, Xiancheng Meng, Xianda Sun, Xiaoyu Jiang, Xunhao Jia, Yao Xia, Yimeng Xu, Yinghan Cui, Yingpeng Chen, Yiwen Ning, Yong Wang, Yuxuan Sun, Zhongsheng Liu, Ming Sun, Cheng Luo, Chen Yang, Han Li, Kun Gai · 8 de julio de 2026

  We present KAT-Coder-V2.5, a coding-focused agentic model trained to act autonomously inside real, executable repositories rather than as a single-turn code generator. Its capability is bottlenecked less by model scale than by the scarcity of reproducible environments, verifiable rewards, and high-v…

• A Definition and Roadmap for World Models

  Xinyuan Chen, Haoyu Guo, Shi Guo, Bingqi Jiang, Chunhua Shen, Xing Shen, Tianfan Xue, Yufei Xue, Mulin Yu, Weinan Zhang, Bin Zhao, Bowen Zhou, Ming Zhou · 8 de julio de 2026

  World models -- internal simulators that learn the structure and dynamics of an environment -- have become one of the most actively debated concepts in AI. From model-based reinforcement learning and video generation to embodied robotics and ultimately, physical AI, researchers across AI subfields a…

• Policy Improvement with Style-Specific Demonstrations

  Lingfeng Li, Yunlong Lu, Yongyi Wang, Wenxin Li · 7 de julio de 2026

  Proficient game agents with diverse play styles enrich the gaming experience and enhance the replay value of games. However, recent advancements in game AI based on reinforcement learning have predominantly focused on improving proficiency, whereas methods based on evolution algorithms generate agen…

• Progress- and Reliability-Oriented Group Policy Optimization for Agentic Reinforcement Learning

  Mingxuan Fan, Peiyang Liu · 7 de julio de 2026

  Group-based reinforcement learning (RL) has become an effective paradigm for improving large language model agents on long-horizon interactive tasks. To obtain finer-grained policy updates than trajectory-level optimization, recent work has moved toward step-level group-based RL, where intermediate …

• GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks

  Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue, Jonathan Francis, Christian Juette, Peter Schaldenbrand, Muhammet Yunus Seker, Ruwan Wickramarachchi, Uksang Yoo, Guanzhi Wang, Adithyavairavan Murali, Balakumar Sundaralingam, S. Shankar Sastry, Spencer Huang, Yuke Zhu, Linxi "Jim" Fan, Ken Goldberg · 7 de julio de 2026

  For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on "Variational Automation" (VA), a class of tasks that have larger variation…

• Multi-Robot Open Adaptive Teaming Across Unseen Environments, Partners, and Scales

  Yang Li, Feng Xue, Fan Mo, Yunhao Liu, Jianhong Wang, Ying Wen, Qingrui Zhang, Shaoshuai Mou, Wei Pan · 7 de julio de 2026

  Deploying robot teams in the real world requires simultaneous adaptation to unseen environments, unknown partners, and varying team sizes, yet existing approaches often address these challenges in isolation under the closed-world assumption of fixed teammates. We formalize this as open adaptive mult…

• Ensemble Elastic DQN: A Step Dependent Ensemble Approach for Reducing Overestimation in Deep Value-Based Reinforcement Learning

  Adrian Ly, Richard Dazeley, Peter Vamplew, Francisco Cruz, Sunil Aryal · 7 de julio de 2026

  Deep Q-Networks (DQN) can suffer from overestimation bias because bootstrapped targets use a maximisation operation over noisy value estimates. Ensemble-based methods and multi-step methods have each been used to improve the stability and sample efficiency of value-based reinforcement learning, but …

• A Technical Survey of Reinforcement Learning Techniques for Large Language Models

  Saksham Sahai Srivastava, Vaneet Aggarwal · 7 de julio de 2026

  This survey offers a comprehensive foundation on the integration of RL with language models, highlighting prominent algorithms such as Proximal Policy Optimization (PPO), Q-Learning, and Actor-Critic methods. Additionally, it provides an extensive technical overview of RL techniques specifically tai…

• Mask-based Predictive Representations for Reinforcement Learning

  Kai Zhao · 7 de julio de 2026

  Vision-based deep reinforcement learning involves dealing with high-dimensional inputs of image information. It is crucial to abstract effective states from high-dimensional image inputs and limited samples for sample-efficient reinforcement learning. To address this challenge, inspired by fields su…

• UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning

  Niu Lian, Alan Chen, Zhehao Yu, Chengzhen Duan, Fazhan Liu, Hui Liu, Pei Fu, Jian Luan, Yaowei Wang, Shu-Tao Xia, Jinpeng Wang · 7 de julio de 2026

  Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction traject…

• Deep Learning for Dynamic Programming with Recursive Utility

  Xianhua Peng, Wu Guo · 7 de julio de 2026

  We propose the first deep learning algorithm, the Certainty Equivalent Learning (CEL) algorithm, for solving high-dimensional discrete-time dynamic programming problems with recursive utility. Dynamic programming with recursive utility is numerically challenging because the recursive utility does no…

• Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusions

  Kyohei Suzuki, onstantinos Slavakis · 7 de julio de 2026

  This work delivers two key contributions: one to efficient feature selection in reinforcement learning (RL), the other to the theory of non-monotone inclusions. On the RL side, the estimation bias inherent in conventional regularization schemes is addressed by augmenting classical least-squares …

• Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI

  Kairos Team, Fei Wang, Shan You, Qiming Zhang, Tao Huang, Zuoyi Fu, Zhisheng Zheng, Yunlong Xi, Feng Lv, Xiaoming Wu, Zeyu Liu, Cong Wan, Pu Li, Ruiqing Yang, Xiaoou Li, Wei Wang, Kangkang Zhu, Yuwei Zhang, Shi Fu, Zheng Zhang, Xiaoning Wu, Xuzeng Fan, Dacheng Tao, Xiaogang Wang · 7 de julio de 2026

  We introduce \textbf{Kairos}, a regret-aware native world-action model stack for Physical AI. Kairos is motivated by the view that a physical world model should not aim to fully simulate all future pixels, but should learn and maintain the information most relevant to embodiment control: object stat…

• Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models

  Donna Vakalis · 7 de julio de 2026

  World-model evaluation for model-based reinforcement learning typically asks whether the learned model predicts reward and value well, which can leave planning-relevant errors in the model's latent rollouts unmeasured. We introduce a complementary diagnostic, operator-on-F, that compares a model's k…