Reinforcement Learning in Robotics

Neueste Paper

• Enhancing Deep Deterministic Policy Gradients on Continuous Control Tasks with Decoupled Prioritized Experience Replay

  Mehmet Efe Lorasdagi, Dogan Can Cicek, Furkan Burak Mutlu, Suleyman Serdar Kozat · 8. Dezember 2025

  Background: Deep Deterministic Policy Gradient-based reinforcement learning algorithms utilize Actor-Critic architectures, where both networks are typically trained using identical batches of replayed transitions. However, the learning objectives and update dynamics of the Actor and Critic differ, r…

• Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversity

  Germ\'an Kruszewski, Pierre Erbacher, Jos Rozen, Marc Dymetman · 8. Dezember 2025

  Reinforcement Learning (RL) has become the de facto standard for tuning LLMs to solve tasks involving reasoning. However, growing evidence shows that models trained in such way often suffer from a significant loss in diversity. We argue that this arises because RL implicitly optimizes the "mode-seek…

• EnhancedRL: An Enhanced-State Reinforcement Learning Algorithm for Multi-Task Fusion in Recommender Systems

  Peng Liu, Cong Xu, Jiawei Zhu, Ming Zhao, Bin Wang · 8. Dezember 2025

  As a key stage of Recommender Systems (RSs), Multi-Task Fusion (MTF) is responsible for merging multiple scores output by Multi-Task Learning (MTL) into a single score, finally determining the recommendation results. Recently, Reinforcement Learning (RL) has been applied to MTF to maximize long-term…

• Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learning

  Zhenpeng Su, Leiyu Pan, Minxuan Lv, Tiehua Mei, Zijia Lin, Yuntao Li, Wenping Hu, Ruiming Tang, Kun Gai, Guorui Zhou · 8. Dezember 2025

  Large language model post-training relies on reinforcement learning to improve model capability and alignment quality. However, the off-policy training paradigm introduces distribution shift, which often pushes the policy beyond the trust region, leading to training instabilities manifested as fluct…

• Bridging Interpretability and Optimization: Provably Attribution-Weighted Actor-Critic in Reproducing Kernel Hilbert Spaces

  Na Li, Hangguan Shan, Wei Ni, Wenjie Zhang, Xinyu Li · 8. Dezember 2025

  Actor-critic (AC) methods are a cornerstone of reinforcement learning (RL) but offer limited interpretability. Current explainable RL methods seldom use state attributions to assist training. Rather, they treat all state features equally, thereby neglecting the heterogeneous impacts of individual st…

• Better World Models Can Lead to Better Post-Training Performance

  Prakhar Gupta, Henry Conklin, Sarah-Jane Leslie, Andrew Lee · 4. Dezember 2025

  In this work we study how explicit world-modeling objectives affect the internal representations and downstream capability of Transformers across different training stages. We use a controlled 2x2x2 Rubik's Cube and ask: (1) how does explicitly pretraining a world model affect the model's latent rep…

• Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimation

  Runze Zhao, Yue Yu, Ruhan Wang, Chunfeng Huang, Dongruo Zhou · 4. Dezember 2025

  Continuous-time reinforcement learning (CTRL) provides a natural framework for sequential decision-making in dynamic environments where interactions evolve continuously over time. While CTRL has shown growing empirical success, its ability to adapt to varying levels of problem difficulty remains poo…

• Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+($\lambda$,$\lambda$))-GA

  Tai Nguyen, Phong Le, Andr\'e Biedenkapp, Carola Doerr, Nguyen Dang · 4. Dezember 2025

  Dynamic Algorithm Configuration (DAC) studies the efficient identification of control policies for parameterized optimization algorithms. Numerous studies have leveraged the robustness of decision-making in Reinforcement Learning (RL) to address the optimization challenges in algorithm configuration…

• AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversification

  Geonwoo Cho, Jaemoon Lee, Jaegyun Im, Subi Lee, Jihwan Lee, Sundong Kim · 4. Dezember 2025

  Skill-based reinforcement learning (SBRL) enables rapid adaptation in environments with sparse rewards by pretraining a skill-conditioned policy. Effective skill learning requires jointly maximizing both exploration and skill diversity. However, existing methods often face challenges in simultaneous…

• Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learning

  Juan Sebastian Rojas, Chi-Guhn Lee · 4. Dezember 2025

  Continual reinforcement learning (continual RL) seeks to formalize the notions of lifelong learning and endless adaptation in RL. In particular, the aim of continual RL is to develop RL agents that can maintain a careful balance between retaining useful information and adapting to new situations. To…

• Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning

  Yichen Li, Chicheng Zhang · 4. Dezember 2025

  Imitation learning (IL) is a paradigm for learning sequential decision making policies from experts, leveraging offline demonstrations, interactive annotations, or both. Recent advances show that when annotation cost is tallied per trajectory, Behavior Cloning (BC) which relies solely on offline dem…

• DVPO: Distributional Value Modeling-based Policy Optimization for LLM Post-Training

  Dingwei Zhu, Zhiheng Xi, Shihan Dou, Yuhui Wang, Sixian Li, Junjie Ye, Honglin Guo, Shichun Liu, Chenhao Huang, Yajie Yang, Junlin Shang, Senjie Jin, Ming Zhang, Jiazheng Zhang, Caishuang Huang, Yunke Zhang, Demei Yan, Yuran Wang, Tao Gui · 4. Dezember 2025

  Reinforcement learning (RL) has shown strong performance in LLM post-training, but real-world deployment often involves noisy or incomplete supervision. In such settings, complex and unreliable supervision signals can destabilize training and harm generalization. While existing approaches such as wo…

• Multi-Agent Reinforcement Learning with Communication-Constrained Priors

  Guang Yang, Tianpei Yang, Jingwen Qiao, Yanqing Wu, Jing Huo, Xingguo Chen, Yang Gao · 4. Dezember 2025

  Communication is one of the effective means to improve the learning of cooperative policy in multi-agent systems. However, in most real-world scenarios, lossy communication is a prevalent issue. Existing multi-agent reinforcement learning with communication, due to their limited scalability and robu…

• SPARK: Stepwise Process-Aware Rewards for Reference-Free Reinforcement Learning

  Salman Rahman, Sruthi Gorantla, Arpit Gupta, Swastik Roy, Nanyun Peng, Yang Liu · 4. Dezember 2025

  Process reward models (PRMs) that provide dense, step-level feedback have shown promise for reinforcement learning, yet their adoption remains limited by the need for expensive step-level annotations or ground truth references. We propose SPARK: a three-stage framework where in the first stage a gen…

• Scaling Internal-State Policy-Gradient Methods for POMDPs

  Douglas Aberdeen, Jonathan Baxter · 4. Dezember 2025

  Policy-gradient methods have received increased attention recently as a mechanism for learning to act in partially observable environments. They have shown promise for problems admitting memoryless policies but have been less successful when memory is required. In this paper we develop several impro…

• Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning

  Franki Nguimatsia Tiofack, Th\'eotime Le Hellard, Fabian Schramm, Nicolas Perrin-Gilbert, Justin Carpentier · 4. Dezember 2025

  Offline reinforcement learning often relies on behavior regularization that enforces policies to remain close to the dataset distribution. However, such approaches fail to distinguish between high-value and low-value actions in their regularization components. We introduce Guided Flow Policy (GFP), …

• FOVA: Offline Federated Reinforcement Learning with Mixed-Quality Data

  Nan Qiao, Sheng Yue, Ju Ren, Yaoxue Zhang · 3. Dezember 2025

  Offline Federated Reinforcement Learning (FRL), a marriage of federated learning and offline reinforcement learning, has attracted increasing interest recently. Albeit with some advancement, we find that the performance of most existing offline FRL methods drops dramatically when provided with mixed…

• Reinforcement Learning: An Overview

  Kevin Murphy · 3. Dezember 2025

  This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based methods, policy-based methods, model-based methods, multi-agent RL, LLMs and RL, and various other topics (e.g., offline RL, hierarchical RL, in…

• Adversarial Diffusion for Robust Reinforcement Learning

  Daniele Foffano, Alessio Russo, Alexandre Proutiere · 3. Dezember 2025

  Robustness to modeling errors and uncertainties remains a central challenge in reinforcement learning (RL). In this work, we address this challenge by leveraging diffusion models to train robust RL policies. Diffusion models have recently gained popularity in model-based RL due to their ability to g…

• Reinforcement Learning in POMDP's via Direct Gradient Ascent

  Jonathan Baxter, Peter L. Bartlett · 3. Dezember 2025

  This paper discusses theoretical and experimental aspects of gradient-based approaches to the direct optimization of policy performance in controlled POMDPs. We introduce GPOMDP, a REINFORCE-like algorithm for estimating an approximation to the gradient of the average reward as a function of the par…

• Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts

  Zhongjian Qiao, Rui Yang, Jiafei Lyu, Xiu Li, Zhongxiang Dai, Zhuoran Yang, Siyang Gao, Shuang Qiu · 3. Dezember 2025

  Single-domain offline reinforcement learning (RL) often suffers from limited data coverage, while cross-domain offline RL handles this issue by leveraging additional data from other domains with dynamics shifts. However, existing studies primarily focus on train-time robustness (handling dynamics sh…

• From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers

  Ching Fang, Kanaka Rajan · 3. Dezember 2025

  Humans and animals show remarkable learning efficiency, adapting to new environments with minimal experience. This capability is not well captured by standard reinforcement learning algorithms that rely on incremental value updates. Rapid adaptation likely depends on episodic memory-- the ability to…

• GoRL: An Algorithm-Agnostic Framework for Online Reinforcement Learning with Generative Policies

  Chubin Zhang, Zhenglin Wan, Feng Chen, Xingrui Yu, Ivor Tsang, Bo An · 3. Dezember 2025

  Reinforcement learning (RL) faces a persistent tension: policies that are stable to optimize are often too simple to represent the multimodal action distributions needed for complex control. Gaussian policies provide tractable likelihoods and smooth gradients, but their unimodal form limits expressi…

• Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations

  Vivek Myers, Bill Chunyuan Zheng, Benjamin Eysenbach, Sergey Levine · 3. Dezember 2025

  Approaches for goal-conditioned reinforcement learning (GCRL) often use learned state representations to extract goal-reaching policies. Two frameworks for representation structure have yielded particularly effective GCRL algorithms: (1) *contrastive representations*, in which methods learn "success…

• Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents

  Jane H. Lee, Baturay Saglam, Spyridon Pougkakiotis, Amin Karbasi, Dionysis Kalogerias · 3. Dezember 2025

  Constrained optimization provides a common framework for dealing with conflicting objectives in reinforcement learning (RL). In most of these settings, the objectives (and constraints) are expressed though the expected accumulated reward. However, this formulation neglects risky or even possibly cat…