Reinforcement Learning in Robotics

Últimos artículos

• BPPO: Binary Prefix Policy Optimization for Efficient GRPO-Style Reasoning RL with Concise Responses

  Qingfei Zhao, Huan Song, Shuyu Tian, Jiawei Shao, Xuelong Li · 28 de mayo de 2026

  Group Relative Policy Optimization (GRPO) is widely used for training reasoning models, but updating all sampled completions in each group incurs substantial cost and can reinforce verbose reasoning trajectories. In this paper, we study whether all completions provide equally useful update signals i…

• Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learning

  Meraj Mammadov, Pedro Zuidberg Dos Martires, Johannes Andreas Stork · 28 de mayo de 2026

  Imitation learning (IL) from a state-based reinforcement learning (RL) policy is a common approach to overcome the curse of dimensionality in complex and high-dimensional observation spaces prevalent in robotics. This paper addresses the irreducible imitation gap that emerges when teacher and studen…

• Learning to Assign Prediction Tasks to Agents with Capacity Constraints

  Shang Wu, Saatvik Kher, Padhraic Smyth · 28 de mayo de 2026

  We address the problem of learning to assign prediction tasks to one agent from a set of available human or AI agents. In particular, we focus on the sequential learning of agent expertise and assignment policies where each agent is constrained to handle a fraction of tasks. We provide a general the…

• Explicit Critic Guidance for Aligning Diffusion Models

  Zhengyang Liang, Qihang Zhang, Ceyuan Yang · 28 de mayo de 2026

  Online reinforcement learning is becoming increasingly important for aligning diffusion models with non-differentiable objectives. However, existing methods still face limitations in assigning fine-grained credit along denoising trajectories and in realizing stable value-based optimization. We propo…

• Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think

  Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali, Flavian Vasile · 28 de mayo de 2026

  Large scale reinforcement learning has become a central tool for improving reasoning in large language models. At this scale, generation is often lagged or asynchronous, so updates are performed on data collected by older policies. This makes learning inherently off-policy. Most existing approaches …

• Commit to the Bit: Reactive Reinforcement Learning Done Right

  Onno Eberhard, Claire Vernade, Michael Muehlebach · 28 de mayo de 2026

  Reinforcement learning algorithms are commonly analyzed (and designed) under the Markov assumption. This is unrealistic, as most environments encountered in practice are either partially observable, or require function approximation that restricts the agent to access non-Markovian state features. We…

• SKILLC: Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment

  Hongxiang Lin, Zhirui Kuai, Erpeng Xue, Lei Wang · 28 de mayo de 2026

  Structured skill prompts improve exploration in long-horizon agentic reinforcement learning (RL). Skill-augmented RL methods retain external skills at inference, while skill-internalization RL methods withdraw them during training to enable autonomous performance. However, existing internalization a…

• You Live More Than Once: Towards Hierarchical Skill Meta-Evolving

  Xujun Li, Kehan Zheng, Mingyuan Zhao, Yize Geng, Jinfeng Zhou, Qi Zhu, Fei Mi, Lifeng Shang, Minlie Huang, Hongning Wang · 28 de mayo de 2026

  Test-time skill evolving is regarded as a new paradigm for enhancing deployed agentic systems. Existing works mainly focus on hard-coded skill evolving strategies or parametric learning that rely on expensive parameter updates in the underlying LLMs. In this paper, we demonstrate that test-time refi…

• Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents

  Suji Kim, Kangsan Kim, Sung Ju Hwang · 28 de mayo de 2026

  Computer-use agents (CUAs) have recently made substantial progress, but deploying a separate large expert for each software domain remains expensive. Small open computer-use agents are more practical specialization targets, but they remain substantially weaker and exhibit uneven domain-specific fail…

• Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning

  Sebastian Sanokowski, Kaustubh Patil · 28 de mayo de 2026

  Diffusion models excel at sampling from complex, unnormalized distributions. In this work, we extend Maximum Entropy Reinforcement Learning (ME-RL) to diffusion processes, enabling sampling from the optimal policy trajectory distribution. By minimizing a tractable upper bound on the reverse KL diver…

• Adaptive Coarse-to-Fine Subgoal Refinement for Long-Horizon Offline Goal-Conditioned Reinforcement Learning

  Kaiqiang Ke, Shenghong He, Chengdong Xu, Yuheng Luo, Xiangyuan Lan, Chao Yu · 28 de mayo de 2026

  Offline goal-conditioned reinforcement learning (GCRL) is challenging in long-horizon tasks, where distant state--goal pairs provide weak supervision and value estimates become vulnerable to accumulated bootstrapping errors. Hierarchical methods mitigate this difficulty by introducing intermediate s…

• Optimal Data Acquisition for Reinforcement Learning: A Large Deviations Perspective

  Mingjie Hu, Jian-Qiang Hu, Enlu Zhou · 28 de mayo de 2026

  Data acquisition efficiency is a central challenge in deploying reinforcement learning in business and healthcare operations, where interactions are costly, slow, and often involve humans in the loop. This paper develops a unified large deviations framework for data acquisition in infinite-horizon r…

• Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration

  Zili Wang, Jiajun Chai, Lin Chen, Xiaohan Wang, Shiming Xiang, Guojun Yin · 28 de mayo de 2026

  Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as the standard paradigm for improving reasoning capability of large language models, while Multi-Token Prediction (MTP) has been a widely adopted module in pretraining. Combining them is a natural approach, yet current RL practices d…

• Variance-Adaptive Optimal Algorithm for Reinforcement Learning with Multinomial Logit Function Approximation

  Wonyoung Kim, Min-Hwan Oh, Garud Iyengar, Assaf Zeevi · 28 de mayo de 2026

  Reinforcement learning with multinomial logistic (MNL) function approximation has become an important framework due to its flexibility and broad applicability. While existing studies have established regret guarantees under worst-case analysis, they do not capture how performance depends on the vari…

• ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing

  Ankush Kadu, Aswanth Krishnan · 28 de mayo de 2026

  We present ReflexGrad, a dual-process architecture for within-episode failure recovery in LLM agents without demonstrations. When agents commit to a wrong approach early and exhaust the step budget, the post-failure trajectory contains the information to escape -- but no published architecture acts …

• Global Policy-Space Response Oracles for Two-Player Zero-Sum Games

  Junyu Zhang, Feihong Yang, Jian Wang, Chao Wang, Xudong Zhang · 28 de mayo de 2026

  The Policy-Space Response Oracles (PSRO) framework scales equilibrium computation to large zero-sum games by iteratively expanding a restricted strategy set using deep reinforcement learning (DRL). A central challenge is to construct, under limited computational budgets, a small strategy population …

• Where Rollouts Begin: Low-Load, High-Leverage First-Token Diversification for RLVR

  Soeun Kim, Albert No · 28 de mayo de 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) trains reasoning models without labeled trajectories, relying on grouped rollouts to expose the policy to alternative reasoning paths and a verifier to score them. Rollout diversity has accordingly emerged as a central bottleneck in RLVR, with mo…

• Restoring the Sweet Spot: Pass-Rate Weighted Self-Distillation for LLM Reasoning

  Zehao Liu, Yuanpu Cao, Jinghui Chen, Vasant G. Honavar · 28 de mayo de 2026

  Self-Distillation Policy Optimization (SDPO) provides dense token-level credit assignment for reinforcement learning with large language models by leveraging the model's own feedback-conditioned predictions as a self-teacher. Unlike GRPO, however, whose group-relative advantage naturally concentrate…

• Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning

  Wendi Li, Shawn Im, Sharon Li · 28 de mayo de 2026

  Agentic large language models are increasingly used to solve real-world tasks by reasoning over goals, invoking tools, and interacting with external environments. Reinforcement learning provides a natural framework for improving these behaviors, and recent agent RL methods have achieved strong resul…

• Falsification-driven reinforcement learning for maritime motion planning

  Marlon M\"uller, Florian Finkeldei, Hanna Krasowski, Murat Arcak, Matthias Althoff · 28 de mayo de 2026

  Compliance with maritime traffic rules is essential for the safe operation of autonomous vessels, yet training reinforcement learning (RL) agents to adhere to them is challenging. The behavior of RL agents is shaped by the training scenarios they encounter, but creating scenarios that capture the co…

• Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach

  Guang-Yuan Hao, Lars van der Laan, Aur\'elien Bibaut, Nathan Kallus · 28 de mayo de 2026

  We study the transfer of rewards learned using inverse reinforcement learning from expert demonstrations in one environment to reinforcement learning in a new, different environment. This arises naturally when demonstrations are collected in a controlled environment. We formulate the problem as a jo…

• Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN

  Mohammad Taufeeque, Aaron David Tucker, Adam Gleave, Adri\`a Garriga-Alonso · 28 de mayo de 2026

  We partially reverse-engineer a convolutional recurrent neural network (RNN) trained with model-free reinforcement learning to play the box-pushing game Sokoban. We find that the RNN stores future moves (plans) as activations in particular channels of the hidden state, which we call path channels. A…

• Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifies

  Sitao Cheng, Xunjian Yin, Ruiwen Zhou, Yuxuan Li, Xinyi Wang, Liangming Pan, William Yang Wang, Victor Zhong · 28 de mayo de 2026

  Does Reinforcement Learning (RL) merely amplify existing skills, or synthesize novel skills? We investigate this question through the lens of Complementary Reasoning: the critical practical capability of integrating internal knowledge with external context, a prerequisite for reliable Continual Lear…

• Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity

  Yiran Pang, Zhen Ni, Xiangnan Zhong · 28 de mayo de 2026

  Federated reinforcement learning (FedRL) enables multiple agents to collaboratively train a global policy without sharing raw data, making it ideal for privacy-sensitive applications. However, FedRL faces challenges in heterogeneous environments where differing state-transition dynamics lead to non-…

• Single-Rollout Hidden-State Dynamics for Training-Free RLVR Data Selection

  Jianghao Wu, Jianfei Cai, Weiqiang Wang, Jin Ye, Daniel F. Schmidt, Yasmeen George · 28 de mayo de 2026

  Reinforcement learning with verifiable rewards (RLVR) can yield large reasoning gains from very few training instances, yet its strong sensitivity to which instances are used makes data selection a central bottleneck. Most existing selection pipelines rely on training-time optimization signals and/o…