Reinforcement Learning in Robotics

Últimos artículos

• Skill-R1: Agent Skill Evolution via Reinforcement Learning

  Yash Vishe, Rohan Surana, Xunyi Jiang, Zihan Huang, Xintong Li, Nikki Lijing Kuang, Tong Yu, Ryan A. Rossi, Jingbo Shang, Julian McAuley, Junda Wu · 12 de mayo de 2026

  Agentic large language models often rely on skills, reusable natural language procedures that guide planning, action, and tool use. In practice, skills are typically improved through prompt engineering or by aligning the task LLM itself, which is costly, model-specific, and often infeasible for clos…

• From Passive Reuse to Active Reasoning: Grounding Large Language Models for Neuro-Symbolic Experience Replay

  Yanan Xiao, Yixiang Tang, Zechen Feng, Lu Jiang, Minghao Yin, Pengyang Wang · 12 de mayo de 2026

  While experience replay is essential for data efficiency in reinforcement learning (RL), standard methods treat the replay buffer as a passive memory system, prioritizing samples based on numerical prediction errors rather than their semantic significance. This approach stands in contrast to human l…

• Beyond Penalization: Diffusion-based Out-of-Distribution Detection and Selective Regularization in Offline Reinforcement Learning

  Qingjun Wang, Hongtu Zhou, Hang Yu, Junqiao Zhao, Yanping Zhao, Chen Ye, Ziqiao Wang, Guang Chen · 12 de mayo de 2026

  Offline reinforcement learning (RL) faces a critical challenge of overestimating the value of out-of-distribution (OOD) actions. Existing methods mitigate this issue by penalizing unseen samples, yet they fail to accurately identify OOD actions and may suppress beneficial exploration beyond the beha…

• Revisiting Mixture Policies in Entropy-Regularized Actor-Critic

  Jiamin He, Samuel Neumann, Jincheng Mei, Adam White, Martha White · 12 de mayo de 2026

  Mixture policies theoretically offer greater flexibility than unimodal policies in continuous action reinforcement learning, but the practical benefits of this complexity remain elusive. Mixture policies are notably absent from most state-of-the-art algorithms, raising a fundamental question: Is the…

• Learning the Preferences of a Learning Agent

  Karim Abdel Sadek, Mark Bedaywi, Rhys Gould, Stuart Russell · 12 de mayo de 2026

  For AI systems to be useful to humans, they must understand and act in accordance with our values and preferences. Since specifying preferences is a hard task, inverse reinforcement learning (IRL) aims to develop methods that allow for inferring preferences from observed behavior. However, IRL assum…

• Path-Coupled Bellman Flows for Distributional Reinforcement Learning

  Boyang Xu, Qing Zou, Siqin Yang, Hao Yan · 12 de mayo de 2026

  Distributional reinforcement learning (DRL) models the full return distribution, but existing finite-support or quantile-based methods rely on projections, while recent flow-based approaches can suffer from \emph{boundary mismatch} at the flow source or from \emph{high-variance} bootstrapping when c…

• Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation

  Ye Yu, Xiaopeng Yuan, Haibo Jin, Heming Liu, Yaoning Yu, Haohan Wang · 12 de mayo de 2026

  Recent advances in LLM agents enable systems that autonomously refine workflows, accumulate reusable skills, self-train their underlying models, and maintain persistent memory. However, we show that such self-evolution is often non-monotonic: adapting to new task distributions can progressively degr…

• Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents

  Zhiyuan Fan, Wenwei Jin, Feng Zhang, Bin Li, Yihong Dong, Yao Hu, Jiawei Li · 12 de mayo de 2026

  Experience-driven self-evolving agents aim to overcome the static nature of large language models by distilling reusable experience from past interactions, thus enabling adaptation to novel tasks at deployment time. This process places substantial demands on the foundation model's capacities for abs…

• Reinforcement Learning for Scalable and Trustworthy Intelligent Systems

  Guangchen Lan · 12 de mayo de 2026

  Reinforcement learning has become a powerful paradigm for improving the capability of intelligent systems, but its practical deployment faces two central challenges. First, reinforcement learning must scale efficiently in distributed environments where communication bandwidth is limited and computat…

• Zero-shot Imitation Learning by Latent Topology Mapping

  Maxwell J. Jacobson, Yexiang Xue · 12 de mayo de 2026

  Imitation learning is effective for training agents when expert demonstrations are available, but collecting demonstrations for every complex task in an environment is costly. We study the long-horizon, goal-conditioned setting where a fixed demonstration dataset contains useful behavior, but not co…

• One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning

  Bowen He, Juncheng Dong, Lin Lin, Xiang Cheng · 12 de mayo de 2026

  A central challenge in reinforcement learning (RL) is to learn models that generalize beyond the tasks on which they are trained, a goal traditionally pursued through multi-task and meta RL. Recently, transformer architectures have emerged as a promising approach, enabling adaptation to new tasks vi…

• Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation

  Yuxuan Jiang, Runchao Li, Shubhashis Roy Dipta, Dawei Li, Zhao Yang · 12 de mayo de 2026

  While recent work in Reinforcement Learning with Verifiable Rewards (RLVR) has shown that a small subset of critical tokens disproportionately drives reasoning gains, an analogous token-level understanding of On-Policy Distillation (OPD) remains largely unexplored. In this work, we investigate high-…

• DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards

  Haoyu Hu, Xuandong Zhao, Xuhai "Orson'' Xu, Nori Jacoby · 12 de mayo de 2026

  Reinforcement learning with verifiable rewards (RLVR) generates hundreds of thousands of tokens per training step, with rollout generation dominating the computational cost. The overall token budget can be controlled along two main dimensions: (i) deciding which prompts to allocate rollouts to, and …

• TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment

  Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Zheng Pan, Xin Li, Lan-Zhe Guo · 12 de mayo de 2026

  On-policy self-distillation (self-OPD) densifies reinforcement learning with verifiable rewards (RLVR) by letting a policy teach itself under privileged context. We find that when this guidance spans the full response, all-token KL spends gradients on mostly redundant positions and amplifies privile…

• Evolutionary Ensemble of Agents

  Zongmin Yu, Liu Yang · 12 de mayo de 2026

  We introduce Evolutionary Ensemble (EvE), a decentralized framework that organizes existing, highly capable coding agents into a live, co-evolving system for algorithmic discovery. Rather than reinventing the wheel within the "LLMs as optimizers" paradigm, EvE fixes the base agent substrate and focu…

• Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization

  Xingyuan Hua, Sheng Yue, Ju Ren · 12 de mayo de 2026

  Recent advancements in agentic test-time scaling allow models to gather environmental feedback before committing to final actions. A key limitation of existing methods is that they typically employ undifferentiated exploration strategies, lacking the ability to adaptively distinguish when exploratio…

• Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMs

  Wenzhi Fang, Liangqi Yuan, Guangchen Lan, Dong-Jun Han, Christopher G. Brinton · 12 de mayo de 2026

  Multi-agent large language model (LLM) systems often rely on a controller to coordinate a pool of heterogeneous models, yet existing controllers are typically limited to one-shot routing: they select a model once and return its output directly. Such routing-only designs provide no mechanism to criti…

• Finite-Time Analysis of MCTS in Continuous POMDP Planning

  Da Kong, Vadim Indelman · 11 de mayo de 2026

  This paper presents a finite-time analysis for Monte Carlo Tree Search (MCTS) in Partially Observable Markov Decision Processes (POMDPs), with probabilistic concentration bounds in both discrete and continuous observation spaces. While MCTS-style solvers such as POMCP achieve empirical success in ma…

• Actor-Critic with Active Importance Sampling

  Majid Molaei, Gabor Paczolay, Matteo Papini, Alberto Maria Metelli, Marcello Restelli · 11 de mayo de 2026

  This paper introduces the Active-Importance-Sampling Actor-Critic (AISAC) algorithm, an extension of the Actor-Critic framework for reducing variance in policy gradient estimation. AISAC optimizes the behavior policy to minimize gradient variance while preserving unbiased gradient estimates. Using i…

• Learning CLI Agents with Structured Action Credit under Selective Observation

  Haoyang Su, Ying Wen · 11 de mayo de 2026

  Command line interface (CLI) agents are emerging as a practical paradigm for agent-computer interaction over evolving filesystems, executable command line programs, and online execution feedback. Recent work has used reinforcement learning (RL) to learn these interaction abilities from verifiable ta…

• Model-Driven Policy Optimization in Differentiable Simulators via Stochastic Exploration

  Yuval Aroosh, Ayal Taitler · 11 de mayo de 2026

  Differentiable planning enables gradient-based optimization of decision-making problems by leveraging differentiable models of system dynamics. However, in highly nonlinear and hybrid discrete-continuous domains, the resulting optimization landscapes are often ill-conditioned, with flat regions and …

• Actor-Critic Algorithm for Dynamic Expectile and CVaR

  Yudong Luo, Erick Delage · 11 de mayo de 2026

  Optimizing dynamic risk with stochastic policies is challenging in both policy updates and value learning. The former typically requires transition perturbation, while the latter may rely on model-based approaches. To address these challenges, we propose a surrogate policy gradient without transitio…

• SLOPE: Optimistic Potential Landscape Shaping for Model-based Reinforcement Learning

  Yao-Hui Li, Zeyu Wang, Xin Li, Wei Pang, Yingfang Yuan, Zhengkun Chen, Boya Zhang, Riashat Islam, Alex Lamb, Yonggang Zhang · 11 de mayo de 2026

  Model-based reinforcement learning (MBRL) is sample-efficient but struggles in sparse reward settings. A critical bottleneck arises from the lack of informative gradients in sparse settings, where standard reward models often yield flat landscapes that struggle to guide planning. To address this cha…

• Multi-Environment POMDPs with Finite-Horizon Objectives

  L\'eonard Brice, Filip Cano, Krishnendu Chatterjee, Thomas A. Henzinger, Stefanie Muroya · 11 de mayo de 2026

  Partially Observable Markov Decision Processes (POMDPs) are systems in which one agent interacts with a stochastic environment, and receives only partial information about the current state. In a multi-environment POMDP (MEPOMDP), the initial state is unknown, and assumed to be adversarially chosen.…

• Reason to Play: Behavioral and Brain Alignment Between Frontier LRMs and Human Game Learners

  Botos Csaba, Sreejan Kumar, Austin Tudor David Andrews, Laurence Hunt, Chris Summerfield, Joshua B. Tenenbaum, Rui Ponte Costa, Marcelo G. Mattar, Momchil Tomov · 11 de mayo de 2026

  Humans rapidly learn abstract knowledge when encountering novel environments and flexibly deploy this knowledge to guide efficient and intelligent action. Can modern AI systems learn and plan in a similar way? We study this question using a dataset of complex human gameplay with concurrent fMRI reco…