Reinforcement Learning in Robotics

Neueste Paper

• LRT-Diffusion: Calibrated Risk-Aware Guidance for Diffusion Policies

  Ximan Sun, Xiang Cheng · 30. Oktober 2025

  Diffusion policies are competitive for offline reinforcement learning (RL) but are typically guided at sampling time by heuristics that lack a statistical notion of risk. We introduce LRT-Diffusion, a risk-aware sampling rule that treats each denoising step as a sequential hypothesis test between th…

• MDPs with a State Sensing Cost

  Vansh Kapoor, Jayakrishnan Nair · 30. Oktober 2025

  In many practical sequential decision-making problems, tracking the state of the environment incurs a sensing/communication/computation cost. In these settings, the agent's interaction with its environment includes the additional component of deciding when to sense the state, in a manner that balanc…

• Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problems

  Christian Walder, Deep Karkhanis · 30. Oktober 2025

  Reinforcement Learning (RL) algorithms sample multiple n>1 solution attempts for each problem and reward them independently. This optimizes for pass@1 performance and prioritizes the strength of isolated samples at the expense of the diversity and collective utility of sets of samples. This under-ut…

• Enhancing Hierarchical Reinforcement Learning through Change Point Detection in Time Series

  Hemanath Arumugam, Falong Fan, Bo Liu · 30. Oktober 2025

  Hierarchical Reinforcement Learning (HRL) enhances the scalability of decision-making in long-horizon tasks by introducing temporal abstraction through options-policies that span multiple timesteps. Despite its theoretical appeal, the practical implementation of HRL suffers from the challenge of aut…

• Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learning

  Sagalpreet Singh, Rishi Saket, Aravindan Raghuveer · 30. Oktober 2025

  Reinforcement Learning algorithms are primarily focused on learning a policy that maximizes expected return. As a result, the learned policy can exploit one or few reward sources. However, in many natural situations, it is desirable to learn a policy that induces a dispersed marginal state distribut…

• Redistributing Rewards Across Time and Agents for Multi-Agent Reinforcement Learning

  Aditya Kapoor, Kale-ab Tessera, Mayank Baranwal, Harshad Khadilkar, Jan Peters, Stefano Albrecht, Mingfei Sun · 30. Oktober 2025

  Credit assignmen, disentangling each agent's contribution to a shared reward, is a critical challenge in cooperative multi-agent reinforcement learning (MARL). To be effective, credit assignment methods must preserve the environment's optimal policy. Some recent approaches attempt this by enforcing …

• The Formalism-Implementation Gap in Reinforcement Learning Research

  Pablo Samuel Castro · 29. Oktober 2025

  The last decade has seen an upswing in interest and adoption of reinforcement learning (RL) techniques, in large part due to its demonstrated capabilities at performing certain tasks at "super-human levels". This has incentivized the community to prioritize research that demonstrates RL agent perfor…

• Fill in the Blanks: Accelerating Q-Learning with a Handful of Demonstrations in Sparse Reward Settings

  Seyed Mahdi Basiri Azad, Joschka Boedecker · 29. Oktober 2025

  Reinforcement learning (RL) in sparse-reward environments remains a significant challenge due to the lack of informative feedback. We propose a simple yet effective method that uses a small number of successful demonstrations to initialize the value function of an RL agent. By precomputing value est…

• Greedy Sampling Is Provably Efficient for RLHF

  Di Wu, Chengshuai Shi, Jing Yang, Cong Shen · 29. Oktober 2025

  Reinforcement Learning from Human Feedback (RLHF) has emerged as a key technique for post-training large language models. Despite its empirical success, the theoretical understanding of RLHF is still limited, as learning the KL-regularized target with only preference feedback poses additional challe…

• Geometry-Inspired Unified Framework for Discounted and Average Reward MDPs

  Arsenii Mustafin, Xinyi Sheng, Dominik Baumann · 29. Oktober 2025

  The theoretical analysis of Markov Decision Processes (MDPs) is commonly split into two cases - the average-reward case and the discounted-reward case - which, while sharing similarities, are typically analyzed separately. In this work, we extend a recently introduced geometric interpretation of MDP…

• Group-in-Group Policy Optimization for LLM Agent Training

  Lang Feng, Zhenghai Xue, Tingcong Liu, Bo An · 29. Oktober 2025

  Recent advances in group-based reinforcement learning (RL) have driven frontier large language models (LLMs) in single-turn tasks like mathematical reasoning. However, their scalability to multi-turn LLM agent training remains limited. Unlike static tasks, agent-environment interactions unfold over …

• Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?

  Haizhong Zheng, Jiawei Zhao, Beidi Chen · 29. Oktober 2025

  Reinforcement learning has been central to recent advances in large language model reasoning, but most algorithms rely on on-policy training that demands fresh rollouts at every update, limiting efficiency and scalability. Asynchronous RL systems alleviate this by decoupling rollout generation from …

• Sample-efficient and Scalable Exploration in Continuous-Time RL

  Klemens Iten, Lenart Treven, Bhavya Sukhija, Florian D\"orfler, Andreas Krause · 29. Oktober 2025

  Reinforcement learning algorithms are typically designed for discrete-time dynamics, even though the underlying real-world control systems are often continuous in time. In this paper, we study the problem of continuous-time reinforcement learning, where the unknown system dynamics are represented us…

• Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning

  Kevin Huang, Rosario Scalise, Cleah Winston, Ayush Agrawal, Yunchu Zhang, Rohan Baijal, Markus Grotz, Byron Boots, Benjamin Burchfiel, Masha Itkina, Paarth Shah, Abhishek Gupta · 28. Oktober 2025

  Imitation learning has proven effective for training robots to perform complex tasks from expert human demonstrations. However, it remains limited by its reliance on high-quality, task-specific data, restricting adaptability to the diverse range of real-world object configurations and scenarios. In …

• FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoning

  Yuyang Ding, Chi Zhang, Juntao Li, Haibin Lin, Xin Liu, Min Zhang · 28. Oktober 2025

  Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for enhancing the reasoning capabilities of large language models (LLMs). In this context, models explore reasoning trajectories and exploit rollouts with correct answers as positive signals for policy optimiza…

• A Temporal Difference Method for Stochastic Continuous Dynamics

  Haruki Settai, Naoya Takeishi, Takehisa Yairi · 28. Oktober 2025

  For continuous systems modeled by dynamical equations such as ODEs and SDEs, Bellman's Principle of Optimality takes the form of the Hamilton-Jacobi-Bellman (HJB) equation, which provides the theoretical target of reinforcement learning (RL). Although recent advances in RL successfully leverage this…

• The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation

  Farid Bagirov, Mikhail Arkhipov, Ksenia Sycheva, Evgeniy Glukhov, Egor Bogomolov · 28. Oktober 2025

  The application of Reinforcement Learning with Verifiable Rewards (RLVR) to mathematical and coding domains has demonstrated significant improvements in the reasoning and problem-solving abilities of Large Language Models. Despite its success in single generation problem solving, the reinforcement l…

• An Information-Theoretic Analysis of Out-of-Distribution Generalization in Meta-Learning with Applications to Meta-RL

  Xingtu Liu · 28. Oktober 2025

  In this work, we study out-of-distribution generalization in meta-learning from an information-theoretic perspective. We focus on two scenarios: (i) when the testing environment mismatches the training environment, and (ii) when the training environment is broader than the testing environment. The f…

• Learning to Reason Efficiently with Discounted Reinforcement Learning

  Alex Ayoub, Kavosh Asadi, Dale Schuurmans, Csaba Szepesv\'ari, Karim Bouyarmane · 28. Oktober 2025

  Large reasoning models (LRMs) often consume excessive tokens, inflating computational cost and latency. We challenge the assumption that longer responses improve accuracy. By penalizing reasoning tokens using a discounted reinforcement learning setup (interpretable as a small token cost) and analyzi…

• Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Review

  Kumater Ter, RexCharles Donatus, Ore-Ofe Ajayi, Daniel Udekwe · 28. Oktober 2025

  Reinforcement learning (RL) has become a foundational approach for enabling intelligent robotic behavior in dynamic and uncertain environments. This work presents an in-depth review of RL principles, advanced deep reinforcement learning (DRL) algorithms, and their integration into robotic and contro…

• GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping

  Jing Wang, Jiajun Liang, Jie Liu, Henglin Liu, Gongye Liu, Jun Zheng, Wanyuan Pang, Ao Ma, Zhenyu Xie, Xintao Wang, Meng Wang, Pengfei Wan, Xiaodan Liang · 28. Oktober 2025

  Recently, GRPO-based reinforcement learning has shown remarkable progress in optimizing flow-matching models, effectively improving their alignment with task-specific rewards. Within these frameworks, the policy update relies on importance-ratio clipping to constrain overconfident positive and negat…

• Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts

  Di Zhang, Xun Wu, Shaohan Huang, Yaru Hao, Li Dong, Zewen Chi, Zhifang Sui, Furu Wei · 28. Oktober 2025

  Recent advances in reinforcement learning (RL) have substantially improved the training of large-scale language models, leading to significant gains in generation quality and reasoning ability. However, most existing research focuses on dense models, while RL training for Mixture-of-Experts (MoE) ar…

• Reinforcement learning-guided optimization of critical current in high-temperature superconductors

  Mouyang Cheng, Qiwei Wan, Bowen Yu, Eunbi Rha, Michael J Landry, Mingda Li · 28. Oktober 2025

  High-temperature superconductors are essential for next-generation energy and quantum technologies, yet their performance is often limited by the critical current density ($J_c$), which is strongly influenced by microstructural defects. Optimizing $J_c$ through defect engineering is challenging due …

• Mitigating Distribution Shift in Model-based Offline RL via Shifts-aware Reward Learning

  Wang Luo, Haoran Li, Zicheng Zhang, Congying Han, Chi Zhou, Jiayu Lv, Tiande Guo · 28. Oktober 2025

  Model-based offline reinforcement learning trains policies using pre-collected datasets and learned environment models, eliminating the need for direct real-world environment interaction. However, this paradigm is inherently challenged by distribution shift~(DS). Existing methods address this issue …

• SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling

  Yixian Zhang, Shu'ang Yu, Tonghe Zhang, Mo Guang, Haojia Hui, Kaiwen Long, Yu Wang, Chao Yu, Wenbo Ding · 28. Oktober 2025

  Training expressive flow-based policies with off-policy reinforcement learning is notoriously unstable due to gradient pathologies in the multi-step action sampling process. We trace this instability to a fundamental connection: the flow rollout is algebraically equivalent to a residual recurrent co…