Reinforcement Learning in Robotics

Últimos artículos

• Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning

  Yongcan Yu, Lingxiao He, Jian Liang, Kuangpu Guo, Meng Wang, Qianlong Xie, Xingxing Wang, Ran He · 24 de abril de 2026

  Test-time reinforcement learning (TTRL) always adapts models at inference time via pseudo-labeling, leaving it vulnerable to spurious optimization signals from label noise. Through an empirical study, we observe that responses with medium consistency form an ambiguity region and constitute the prima…

• CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning

  Zhenpeng Su, Leiyu Pan, Minxuan Lv, Yuntao Li, Wenping Hu, Fuzheng Zhang, Kun Gai, Guorui Zhou · 24 de abril de 2026

  Reinforcement learning (RL) has become a powerful paradigm for optimizing large language models (LLMs) to handle complex reasoning tasks. A core challenge in this process lies in managing policy entropy, which reflects the balance between exploration and exploitation during training. Existing method…

• Dynamical Priors as a Training Objective in Reinforcement Learning

  Sukesh Subaharan · 24 de abril de 2026

  Standard reinforcement learning (RL) optimizes policies for reward but imposes few constraints on how decisions evolve over time. As a result, policies may achieve high performance while exhibiting temporally incoherent behavior such as abrupt confidence shifts, oscillations, or degenerate inactivit…

• LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent

  Wanli Li, Bince Qu, Bo Pan, Jianyu Zhang, Zheng Liu, Pan Zhang, Wei Chen, Bo Zhang · 24 de abril de 2026

  Reinforcement Learning (RL) has emerged as a powerful training paradigm for LLM-based agents. However, scaling agentic RL for deep research remains constrained by two coupled challenges: hand-crafted synthetic data fails to elicit genuine real-world search capabilities, and real-world search depende…

• Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own

  Weirui Ye, Yunsheng Zhang, Haoyang Weng, Xianfan Gu, Shengjie Wang, Tong Zhang, Mengchen Wang, Pieter Abbeel, Yang Gao · 24 de abril de 2026

  Reinforcement learning (RL) is a promising approach for solving robotic manipulation tasks. However, it is challenging to apply the RL algorithms directly in the real world. For one thing, RL is data-intensive and typically requires millions of interactions with environments, which are impractical i…

• Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrization

  Kevin Godin-Dubois, Anil Yaman, Anna V. Kononova · 24 de abril de 2026

  While Central Pattern Generators (CPGs) and Multi-Layer Perceptrons (MLP) are widely used paradigms in robot control, few systematic studies have been performed on the relative merits of large parameter spaces. In contexts where input and output spaces are small and performance is bounded, having mo…

• Strategic Scaling of Test-Time Compute: A Bandit Learning Approach

  Bowen Zuo, Yinglun Zhu · 24 de abril de 2026

  Scaling test-time compute has emerged as an effective strategy for improving the performance of large language models. However, existing methods typically allocate compute uniformly across all queries, overlooking variation in query difficulty. To address this inefficiency, we formulate test-time co…

• Self-Guided Plan Extraction for Instruction-Following Tasks with Goal-Conditional Reinforcement Learning

  Zoya Volovikova, Nikita Sorokin, Dmitriy Lukashevskiy, Aleksandr Panov, Alexey Skrynnik · 24 de abril de 2026

  We introduce SuperIgor, a framework for instruction-following tasks. Unlike prior methods that rely on predefined subtasks, SuperIgor enables a language model to generate and refine high-level plans through a self-learning mechanism, reducing the need for manual dataset annotation. Our approach invo…

• CubeDAgger: Interactive Imitation Learning for Dynamic Systems with Efficient yet Low-risk Interaction

  Taisuke Kobayashi · 23 de abril de 2026

  Interactive imitation learning makes an agent's control policy robust by stepwise supervisions from an expert. The recent algorithms mostly employ expert-agent switching systems to reduce the expert's burden by limitedly selecting the supervision timing. However, this approach is useful only for sta…

• Lever: Inference-Time Policy Reuse under Support Constraints

  Ihor Vitenki, Noha Ibrahim, Sihem Amer-Yahia · 23 de abril de 2026

  Reinforcement learning (RL) policies are typically trained for fixed objectives, making reuse difficult when task requirements change. We study inference-time policy reuse: given a library of pre-trained policies and a new composite objective, can a high-quality policy be constructed entirely offlin…

• Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning

  Aravind Venugopal, Jiayu Chen, Xudong Wu, Chongyi Zheng, Benjamin Eysenbach, Jeff Schneider · 23 de abril de 2026

  The temporal lag between actions and their long-term consequences makes credit assignment a challenge when learning goal-directed behaviors from data. Generative world models capture the distribution of future states an agent may visit, indicating that they have captured temporal information. How ca…

• Distributional Value Estimation Without Target Networks for Robust Quality-Diversity

  Behrad Koohy, Jamie Bayne · 23 de abril de 2026

  Quality-Diversity (QD) algorithms excel at discovering diverse repertoires of skills, but are hindered by poor sample efficiency and often require tens of millions of environment steps to solve complex locomotion tasks. Recent advances in Reinforcement Learning (RL) have shown that high Update-to-Da…

• From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generation

  Ziwei Huang, Ying Shu, Hao Fang, Quanyu Long, Wenya Wang, Qiushi Guo, Tiezheng Ge, Leilei Gan · 23 de abril de 2026

  Subject-driven image generation models face a fundamental trade-off between identity preservation (fidelity) and prompt adherence (editability). While online reinforcement learning (RL), specifically GPRO, offers a promising solution, we find that a naive application of GRPO leads to competitive deg…

• Near-Future Policy Optimization

  Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Dingyu Yao, Zheng Lin, Peng Fu, Nan Duan, Jiaqi Wang · 23 de abril de 2026

  Reinforcement learning with verifiable rewards (RLVR) has become a core post-training recipe. Introducing suitable off-policy trajectories into on-policy exploration accelerates RLVR convergence and raises the performance ceiling, yet finding a source of such trajectories remains the key challenge. …

• Distributional Inverse Reinforcement Learning

  Feiyang Wu, Ye Zhao, Anqi Wu · 23 de abril de 2026

  We propose a distributional framework for offline Inverse Reinforcement Learning (IRL) that jointly models uncertainty over reward functions and full distributions of returns. Unlike conventional IRL approaches that recover a deterministic reward estimate or match only expected returns, our method c…

• Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization

  Carter Adams, Rafael Oliveira, Gabriel Almeida, Sofia Torres · 23 de abril de 2026

  Reinforcement fine-tuning with verifiable rewards (RLVR) has emerged as a powerful paradigm for equipping large vision-language models (LVLMs) with agentic capabilities such as tool use and multi-step reasoning. Despite striking empirical successes, most notably Visual Agentic Reinforcement Fine-Tun…

• Supplement Generation Training for Enhancing Agentic Task Performance

  Young Min Cho, Daniele Bonadiman, Divya Bhargavi, Tamer Alkhouli, Salvatore Romeo, Dongwei Jiang, Khushbu Pahwa, Yubin Ge, Etsuko Ishii, Monica Sunkara, Yi Zhang · 23 de abril de 2026

  Training large foundation models for agentic tasks is increasingly impractical due to the high computational costs, long iteration cycles, and rapid obsolescence as new models are continuously released. Instead of post-training massive models for every new task or domain, we propose Supplement Gener…

• Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

  Julien Audiffren, Michal Valko, Alessandro Lazaric, Mohammad Ghavamzadeh · 23 de abril de 2026

  A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possib…

• Planning in entropy-regularized Markov decision processes and games

  Jean-Bastien Grill, Omar Darwiche Domingues, Pierre M\'enard, R\'emi Munos, Michal Valko · 22 de abril de 2026

  We propose SmoothCruiser, a new planning algorithm for estimating the value function in entropy-regularized Markov decision processes and two-player games, given a generative model of the environment. SmoothCruiser makes use of the smoothness of the Bellman operator promoted by the regularization to…

• Beyond the Bellman Fixed Point: Geometry and Fast Policy Identification in Value Iteration

  Donghwan Lee · 22 de abril de 2026

  Dynamic programming is one of the most fundamental methodologies for solving Markov decision problems. Among its many variants, Q-value iteration (Q-VI) is particularly important due to its conceptual simplicity and its classical contraction-based convergence guarantee. Despite the central role of t…

• Accelerating trajectory optimization with Sobolev-trained diffusion policies

  Th\'eotime Le Hellard, Franki Nguimatsia Tiofack, Quentin Le Lidec, Justin Carpentier · 22 de abril de 2026

  Trajectory Optimization (TO) solvers exploit known system dynamics to compute locally optimal trajectories through iterative improvements. A downside is that each new problem instance is solved independently; therefore, convergence speed and quality of the solution found depend on the initial trajec…

• Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback

  Qiang Liu, Adrienne Kline, Ermin Wei · 22 de abril de 2026

  Safe Reinforcement Learning from Human Feedback (Safe RLHF) has recently achieved empirical success in developing helpful and harmless large language models by decoupling human preferences regarding helpfulness and harmlessness. Existing approaches typically rely on fitting fixed horizon reward mode…

• Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning

  Linwei Dong, Ruoyu Guo, Ge Bai, Zehuan Yuan, Yawei Luo, Changqing Zou · 22 de abril de 2026

  Diffusion distillation, exemplified by Distribution Matching Distillation (DMD), has shown great promise in few-step generation but often sacrifices quality for sampling speed. While integrating Reinforcement Learning (RL) into distillation offers potential, a naive fusion of these two objectives re…

• FASTER: Value-Guided Sampling for Fast RL

  Perry Dong, Alexander Swerdlow, Dorsa Sadigh, Chelsea Finn · 22 de abril de 2026

  Some of the most performant reinforcement learning algorithms today can be prohibitively expensive as they use test-time scaling methods such as sampling multiple action candidates and selecting the best one. In this work, we propose FASTER, a method for getting the benefits of sampling-based test-t…

• Safe Continual Reinforcement Learning in Non-stationary Environments

  Austin Coursey, Abel Diaz-Gonzalez, Marcos Quinones-Grueiro, Gautam Biswas · 22 de abril de 2026

  Reinforcement learning (RL) offers a compelling data-driven paradigm for synthesizing controllers for complex systems when accurate physical models are unavailable; however, most existing control-oriented RL methods assume stationarity and, therefore, struggle in real-world non-stationary deployment…