Reinforcement Learning in Robotics

Últimos artículos

• Value Explicit Pretraining for Learning Transferable Representations

  Kiran Lekkala, Henghui Bao, Sumedh A. Sontakke, Erdem Biyik, Laurent Itti · 4 de mayo de 2026

  Understanding visual inputs for a given task amidst varied changes is a key challenge posed by visual reinforcement learning agents. We propose \textit{Value Explicit Pretraining} (VEP), a method that learns generalizable representations for transfer reinforcement learning. VEP enables efficient lea…

• SAVGO: Learning State-Action Value Geometry with Cosine Similarity for Continuous Control

  Stavros Orfanoudakis, Pedro P. Vergara · 4 de mayo de 2026

  While representation and similarity learning have improved the sample efficiency of Reinforcement Learning (RL), they are rarely used to shape policy updates directly in the action space. To bridge this gap, a geometry-aware RL algorithm that explicitly incorporates value-based similarity into the p…

• Model-Based Reinforcement Learning with Double Oracle Efficiency in Policy Optimization and Offline Estimation

  Haichen Hu, Jian Qian, David Simchi-Levi · 4 de mayo de 2026

  Reinforcement learning (RL) in large environments often suffers from severe computational bottlenecks, as conventional regret minimization algorithms require repeated, costly calls to planning and statistical estimation oracles. While recent advances have explored offline oracle-efficient algorithms…

• PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning

  Feijie Wu, Weiwu Zhu, Yuxiang Zhang, Soumya Chatterjee, Jiarong Zhu, Fan Mo, Rong Luo, Jing Gao · 4 de mayo de 2026

  Multi-tool-integrated reasoning enables LLM-empowered tool-use agents to solve complex tasks by interleaving natural-language reasoning with calls to external tools. However, training such agents from outcome-only rewards suffers from credit-assignment ambiguity, obscuring which intermediate tool-us…

• AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning

  Haotian Zhao, Yuxin Zhang, Songlin Zhou, Stephen S. -T. Yau, Wenyu Zhang, Lun Tian, Tianshu Zhu, Yifeng Huang, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu · 4 de mayo de 2026

  Reinforcement learning (RL) has significantly advanced the ability of large language model (LLM) agents to interact with environments and solve multi-turn tasks. Yet effective training remains challenging, as sparse, outcome-only rewards make it difficult to assign credit to individual steps in an a…

• Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity

  Anamika Lochab, Bolian Li, Ruqi Zhang · 4 de mayo de 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) has achieved substantial gains in single-attempt accuracy (Pass@1) on reasoning tasks, yet often suffers from reduced multi-sample coverage (Pass@K), indicating diversity collapse. We identify a structural cause for this degradation: common RLVR …

• Augmented Lagrangian Multiplier Network for State-wise Safety in Reinforcement Learning

  Jiaming Zhang, Yujie Yang, Yao Lyu, Shengbo Eben Li, Liping Zhang · 4 de mayo de 2026

  Safety is a primary challenge in real-world reinforcement learning (RL). Formulating safety requirements as state-wise constraints has become a prominent paradigm. Handling state-wise constraints with the Lagrangian method requires a distinct multiplier for every state, necessitating neural networks…

• Long-Horizon Model-Based Offline Reinforcement Learning Without Explicit Conservatism

  Tianwei Ni, Esther Derman, Vineet Jain, Vincent Taboga, Siamak Ravanbakhsh, Pierre-Luc Bacon · 4 de mayo de 2026

  Popular offline reinforcement learning (RL) methods rely on explicit conservatism, penalizing out-of-dataset actions or restricting rollout horizons. We question the universality of this principle and revisit a complementary Bayesian perspective for test-time adaptation. By modeling a posterior over…

• Pessimism-Free Offline Learning in General-Sum Games via KL Regularization

  Claire Chen, Yuheng Zhang · 4 de mayo de 2026

  Offline multi-agent reinforcement learning in general-sum settings is challenged by the distribution shift between logged datasets and target equilibrium policies. While standard methods rely on manual pessimistic penalties, we demonstrate that KL regularization suffices to stabilize learning and ac…

• Data Deletion Can Help in Adaptive RL

  Param Budhraja, Aditya Gangrade, Alex Olshevsky, Venkatesh Saligrama · 4 de mayo de 2026

  Deploying reinforcement learning policies in the real world requires adapting to time-varying environments. We study this problem in the contextual Markov Decision Process (cMDP) framework, where a family of environments is indexed by a low-dimensional context unknown at test time. The standard appr…

• Global Optimality for Constrained Exploration via Penalty Regularization

  Florian Wolf, Ilyas Fatkhullin, Niao He · 1 de mayo de 2026

  Efficient exploration is a central problem in reinforcement learning and is often formalized as maximizing the entropy of the state-action occupancy measure. While unconstrained maximum-entropy exploration is relatively well understood, real-world exploration is often constrained by safety, resource…

• Detecting is Easy, Adapting is Hard: Local Expert Growth for Visual Model-Based Reinforcement Learning under Distribution Shift

  Haiyang Zhao · 1 de mayo de 2026

  Visual model-based reinforcement learning (MBRL) agents can perform well on the training distribution, but often break down once the test environment shifts. In visual MBRL, recognizing that a shift has occurred is often the easier part; the harder part is turning that recognition into useful action…

• Cost-Aware Learning

  Clara Mohri, Amir Globerson, Haim Kaplan, Tomer Koren, Yishay Mansour · 1 de mayo de 2026

  We consider the problem of Cost-Aware Learning, where sampling different component functions of a finite-sum objective incurs different costs. The objective is to reach a target error while minimizing the total cost. First, we propose the Cost-Aware Stochastic Gradient Descent algorithm for convex f…

• GUI Agents with Reinforcement Learning: Toward Digital Inhabitants

  Junan Hu, Jian Liu, Jingxiang Lai, Jiarui Hu, Yiwei Sheng, Shuang Chen, Jian Li, Dazhao Du, Song Guo · 1 de mayo de 2026

  Graphical User Interface (GUI) agents have emerged as a promising paradigm for intelligent systems that perceive and interact with graphical interfaces visually. Yet supervised fine-tuning alone cannot handle long-horizon credit assignment, distribution shifts, and safe exploration in irreversible e…

• A High-Throughput Compute-Efficient POMDP Hide-And-Seek-Engine (HASE) for Multi-Agent Operations

  Timothy Flavin, Sandip Sen · 1 de mayo de 2026

  Reinforcement Learning (RL) algorithms exhibit high sample complexity, particularly when applied to Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). As a response, projects such as SampleFactory, EnvPool, Brax, and IsaacLab migrate parallel execution of classic environments…

• Accelerating Policy Synthesis in Large-Scale MDPs via Hierarchical Adaptive Refinement

  Alexandros Evangelidis, Gricel V\'azquez, Simos Gerasimou · 1 de mayo de 2026

  Software-intensive systems, such as software product lines and robotics, utilise Markov decision processes (MDPs) to capture uncertainty and analyse sequential decision-making problems. Despite the usefulness of conventional policy synthesis methods, they fail to scale to large state spaces. Our app…

• RHyVE: Competence-Aware Verification and Phase-Aware Deployment for LLM-Generated Reward Hypotheses

  Feiyu Wu, Xu Zheng, Zhuocheng Wang, Yi ming Dai, Hui Li · 1 de mayo de 2026

  Large language models (LLMs) make reward design in reinforcement learning substantially more scalable, but generated rewards are not automatically reliable training objectives. Existing work has focused primarily on generating, evolving, or selecting reward candidates, while paying less attention to…

• PRISM: Pre-alignment via Black-box On-policy Distillation for Multimodal Reinforcement Learning

  Sudong Wang, Weiquan Huang, Xiaomin Yu, Zuhao Yang, Hehai Lin, Keming Wu, Chaojun Xiao, Chen Chen, Wenxuan Wang, Beier Zhu, Yunjian Zhang, Chengwei Qin · 1 de mayo de 2026

  The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR). However, SFT introduces distributional drift that neither preserves the model's original capabilities…

• EXPO: Stable Reinforcement Learning with Expressive Policies

  Perry Dong, Qiyang Li, Dorsa Sadigh, Chelsea Finn · 1 de mayo de 2026

  We study the problem of training and fine-tuning expressive policies with online reinforcement learning (RL) given an offline dataset. Training expressive policy classes with online RL present a unique challenge of stable value maximization. Unlike simpler Gaussian policies commonly used in online R…

• Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning

  Jingcheng Deng, Zihao Wei, Liang Pang, Junhong Wu, Shicheng Xu, Zenghao Duan, Huawei Shen · 1 de mayo de 2026

  Latent reasoning offers a more efficient alternative to explicit reasoning by compressing intermediate reasoning into continuous representations and substantially shortening reasoning chains. However, existing latent reasoning methods mainly focus on supervised learning, and reinforcement learning i…

• Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning

  Shijin Gong, Kai Ye, Jin Zhu, Xinyu Zhang, Hongyi Zhou, Chengchun Shi · 1 de mayo de 2026

  Recent advances in large language models (LLMs) have increasingly relied on reinforcement learning (RL) to improve their reasoning capabilities. Three approaches have been widely adopted: (i) Proximal policy optimization and advantage actor-critic rely on a deep neural network to estimate the value …

• Rethinking Agentic Reinforcement Learning In Large Language Models

  Fangming Cui, Ruixiao Zhu, Cheng Fang, Sunan Li, Jiahong Li · 1 de mayo de 2026

  Reinforcement Learning (RL) has traditionally focused on training specialized agents to optimize predefined reward functions within narrowly defined environments. However, the advent of powerful Large Language Models (LLMs) and increasingly complex, open-ended tasks has catalyzed a paradigm shift to…

• Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents

  Anh Ta, Junjie Zhu, Shahin Shayandeh · 1 de mayo de 2026

  Tool-calling agents are evaluated on tool selection, parameter accuracy, and scope recognition, yet LLM trajectory assessments remain inherently post-hoc. Disconnected from the active execution loop, such assessments identify errors that are usually addressed through prompt-tuning or retraining, and…

• Bayesian policy gradient and actor-critic algorithms

  Mohammad Ghavamzadeh, Yaakov Engel, Michal Valko · 1 de mayo de 2026

  Policy gradient methods are reinforcement learning algorithms that adapt a parameterized policy by following a performance gradient estimate. Conventional policy gradient methods use Monte-Carlo techniques to estimate the gradient, which tend to have high variance, requiring many samples and resulti…

• Can Tabular Foundation Models Guide Exploration in Robot Policy Learning?

  Buqing Ou, Frederike D\"umbgen · 1 de mayo de 2026

  Policy optimization in high-dimensional continuous control for robotics remains a challenging problem. Predominant methods are inherently local and often require extensive tuning and carefully chosen initial guesses for good performance, whereas more global and less initialization-sensitive search m…