Reinforcement Learning in Robotics

Neueste Paper

• Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes

  Alessandro Trapasso, Luca Iocchi, Fabio Patrizi · 17. Dezember 2025

  Many practical decision-making problems involve tasks whose success depends on the entire system history, rather than on achieving a state with desired properties. Markovian Reinforcement Learning (RL) approaches are not suitable for such tasks, while RL with non-Markovian reward decision processes …

• Understanding and Improving Hyperbolic Deep Reinforcement Learning

  Timo Klein, Thomas Lang, Andrii Shkabrii, Alexander Sturm, Kevin Sidak, Lukas Miklautz, Claudia Plant, Yllka Velaj, Sebastian Tschiatschek · 17. Dezember 2025

  The performance of reinforcement learning (RL) agents depends critically on the quality of the underlying feature representations. Hyperbolic feature spaces are well-suited for this purpose, as they naturally capture hierarchical and relational structure often present in complex RL environments. How…

• Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning

  Wei Duan, Jie Lu, Junyu Xuan · 17. Dezember 2025

  In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained communication over fixed physical graphs. Existing methods often assume static neighborhoods, limiting adaptability to dynamic or heterogeneous environments. Whil…

• Sim2Real Reinforcement Learning for Soccer skills

  Jonathan Spraggett · 17. Dezember 2025

  This thesis work presents a more efficient and effective approach to training control-related tasks for humanoid robots using Reinforcement Learning (RL). The traditional RL methods are limited in adapting to real-world environments, complexity, and natural motions, but the proposed approach overcom…

• Reliable Policy Iteration: Performance Robustness Across Architecture and Environment Perturbations

  S. R. Eshwar, Aniruddha Mukherjee, Kintan Saha, Krishna Agarwal, Gugan Thoppe, Aditya Gopalan, Gal Dalal · 16. Dezember 2025

  In a recent work, we proposed Reliable Policy Iteration (RPI), that restores policy iteration's monotonicity-of-value-estimates property to the function approximation setting. Here, we assess the robustness of RPI's empirical performance on two classical control tasks -- CartPole and Inverted Pendul…

• Reliability-Adjusted Prioritized Experience Replay

  Leonard S. Pleiss, Tobias Sutter, Maximilian Schiffer · 16. Dezember 2025

  Experience replay enables data-efficient learning from past experiences in online reinforcement learning agents. Traditionally, experiences were sampled uniformly from a replay buffer, regardless of differences in experience-specific learning potential. In an effort to sample more efficiently, resea…

• ADHint: Adaptive Hints with Difficulty Priors for Reinforcement Learning

  Feng Zhang, Zezhong Tan, Xinhong Ma, Ziqiang Dong, Xi Leng, Jianfei Zhao, Xin Sun, Yang Yang · 16. Dezember 2025

  To combine the advantages of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), recent methods have integrated ''hints'' into post-training, which are prefix segments of complete reasoning trajectories, aiming for powerful knowledge expansion and reasoning generalization. However, existin…

• Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection

  Zihui Zhao, Zechang Li · 16. Dezember 2025

  Direct Preference Optimization (DPO) has emerged as a lightweight and effective alternative to Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning with AI Feedback (RLAIF) for aligning large language and vision-language models. However, the standard DPO formulation, in which…

• Differentiable Evolutionary Reinforcement Learning

  Sitao Cheng, Tianle Li, Xuhan Huang, Xunjian Yin, Difan Zou · 16. Dezember 2025

  The design of effective reward functions presents a central and often arduous challenge in reinforcement learning (RL), particularly when developing autonomous agents for complex reasoning tasks. While automated reward optimization approaches exist, they typically rely on derivative-free evolutionar…

• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models

  Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng-Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping · 16. Dezember 2025

  Building general-purpose reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training cur…

• OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning

  Guanhua Ji, Harsha Polavaram, Lawrence Yunliang Chen, Sandeep Bajamahal, Zehan Ma, Simeon Adebola, Chenfeng Xu, Ken Goldberg · 16. Dezember 2025

  Large and diverse datasets are needed for training generalist robot policies that have potential to control a variety of robot embodiments -- robot arm and gripper combinations -- across diverse tasks and environments. As re-collecting demonstrations and retraining for each new hardware platform are…

• SACn: Soft Actor-Critic with n-step Returns

  Jakub {\L}yskawa, Jakub Lewandowski, Pawe{\l} Wawrzy\'nski · 16. Dezember 2025

  Soft Actor-Critic (SAC) is widely used in practical applications and is now one of the most relevant off-policy online model-free reinforcement learning (RL) methods. The technique of n-step returns is known to increase the convergence speed of RL algorithms compared to their 1-step returns-based ve…

• B\'ezierFlow: B\'ezier Stochastic Interpolant Schedulers for Few-Step Generation

  Yunhong Min, Juil Koo, Seungwoo Yoo, Minhyuk Sung · 16. Dezember 2025

  We introduce B\'ezierFlow, a lightweight training approach for few-step generation with pretrained diffusion and flow models. B\'ezierFlow achieves a 2-3x performance improvement for sampling with $\leq$ 10 NFEs while requiring only 15 minutes of training. Recent lightweight training approaches have…

• Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning

  Wei Duan, Jie Lu, En Yu, Junyu Xuan · 15. Dezember 2025

  Graph-based multi-agent reinforcement learning (MARL) enables coordinated behavior under partial observability by modeling agents as nodes and communication links as edges. While recent methods excel at learning sparse coordination graphs-determining who communicates with whom-they do not address wh…

• Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes

  Junghwan Lim, Sungmin Lee, Dongseok Kim, Taehyun Kim, Eunhwan Park, Jeesoo Lee, Jeongdoo Lee, Junhyeok Lee, Wai Ting Cheung, Dahye Choi, Minsu Ha, Jaeheui Her, Jaeyeon Huh, Hanbin Jung, Changjin Kang, Beomgyu Kim, Minjae Kim, Taewhan Kim, Youngrok Kim, Hyukjin Kweon, Haesol Lee, Kungyu Lee, Dongpin Oh, Yeongjae Park, Bokki Ryu, Dongjoo Weon · 15. Dezember 2025

  We introduce Motif-2-12.7B-Reasoning, a 12.7B parameter language model designed to bridge the gap between open-weight systems and proprietary frontier models in complex reasoning and long-context understanding. Addressing the common challenges of model collapse and training instability in reasoning …

• Memoryless Policy Iteration for Episodic POMDPs

  Roy van Zuijlen, Duarte Antunes · 15. Dezember 2025

  Memoryless and finite-memory policies offer a practical alternative for solving partially observable Markov decision processes (POMDPs), as they operate directly in the output space rather than in the high-dimensional belief space. However, extending classical methods such as policy iteration to thi…

• A-LAMP: Agentic LLM-Based Framework for Automated MDP Modeling and Policy Generation

  Hong Je-Gal, Chan-Bin Yi, Hyun-Suk Lee · 15. Dezember 2025

  Applying reinforcement learning (RL) to real-world tasks requires converting informal descriptions into a formal Markov decision process (MDP), implementing an executable environment, and training a policy agent. Automating this process is challenging due to modeling errors, fragile code, and misali…

• Rethinking Expert Trajectory Utilization in LLM Post-training

  Bowen Ding, Yuhan Chen, Jiayang Lv, Jiyao Yuan, Qi Zhu, Shuangshuang Tian, Dantong Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin · 15. Dezember 2025

  While effective post-training integrates Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), the optimal mechanism for utilizing expert trajectories remains unresolved. We propose the Plasticity-Ceiling Framework to theoretically ground this landscape, decomposing performance into foundati…

• Agile Flight Emerges from Multi-Agent Competitive Racing

  Vineet Pasumarti, Lorenzo Bianchi, Antonio Loquercio · 15. Dezember 2025

  Through multi-agent competition and the sparse high-level objective of winning a race, we find that both agile flight (e.g., high-speed motion pushing the platform to its physical limits) and strategy (e.g., overtaking or blocking) emerge from agents trained with reinforcement learning. We provide e…

• Aligning Humans and Robots via Reinforcement Learning from Implicit Human Feedback

  Suzie Kim, Hye-Bin Shin, Seong-Whan Lee · 15. Dezember 2025

  Conventional reinforcement learning (RL) ap proaches often struggle to learn effective policies under sparse reward conditions, necessitating the manual design of complex, task-specific reward functions. To address this limitation, rein forcement learning from human feedback (RLHF) has emerged as a …

• The Emergence of Complex Behavior in Large-Scale Ecological Environments

  Joseph Bejjani, Chase Van Amburg, Chengrui Wang, Chloe Huangyuan Su, Sarah M. Pratt, Yasin Mazloumi, Naeem Khoshnevis, Sham M. Kakade, Kiant\'e Brantley, Aaron Walsman · 15. Dezember 2025

  We explore how physical scale and population size shape the emergence of complex behaviors in open-ended ecological environments. In our setting, agents are unsupervised and have no explicit rewards or learning objectives but instead evolve over time according to reproduction, mutation, and selectio…

• Decoupled Q-Chunking

  Qiyang Li, Seohong Park, Sergey Levine · 12. Dezember 2025

  Temporal-difference (TD) methods learn state and action values efficiently by bootstrapping from their own future value predictions, but such a self-bootstrapping mechanism is prone to bootstrapping bias, where the errors in the value targets accumulate across steps and result in biased value estima…

• SEMDICE: Off-policy State Entropy Maximization via Stationary Distribution Correction Estimation

  Jongmin Lee, Meiqi Sun, Pieter Abbeel · 12. Dezember 2025

  In the unsupervised pre-training for reinforcement learning, the agent aims to learn a prior policy for downstream tasks without relying on task-specific reward functions. We focus on state entropy maximization (SEM), where the goal is to learn a policy that maximizes the entropy of the state statio…

• Multi-Objective Reward and Preference Optimization: Theory and Algorithms

  Akhil Agnihotri · 12. Dezember 2025

  This thesis develops theoretical frameworks and algorithms that advance constrained reinforcement learning (RL) across control, preference learning, and alignment of large language models. The first contribution addresses constrained Markov Decision Processes (CMDPs) under the average-cost criterion…

• Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain

  Shengbo Wang, Nian Si · 12. Dezember 2025

  Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algorithms, and applications focus on finite-horizon or discounted models. Long-run average-reward formulations, while natural in many operations research and mana…