Reinforcement Learning in Robotics

Neueste Paper

• InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning

  Yuchen Yan, Liang Jiang, Jin Jiang, Shuaicheng Li, Zujie Wen, Zhiqiang Zhang, Jun Zhou, Jian Shao, Yueting Zhuang, Yongliang Shen · 9. Februar 2026

  Large reasoning models achieve strong performance by scaling inference-time chain-of-thought, but this paradigm suffers from quadratic cost, context length limits, and degraded reasoning due to lost-in-the-middle effects. Iterative reasoning mitigates these issues by periodically summarizing interme…

• On the Plasticity and Stability for Post-Training Large Language Models

  Wenwen Qiang, Ziyin Gu, Jiahuan Zhou, Jie Hu, Jingyao Wang, Changwen Zheng, Hui Xiong · 9. Februar 2026

  Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads t…

• Flow Matching for Offline Reinforcement Learning with Discrete Actions

  Fairoz Nower Khan, Nabuat Zaman Nahim, Ruiquan Huang, Haibo Yang, Peizhong Ju · 9. Februar 2026

  Generative policies based on diffusion models and flow matching have shown strong promise for offline reinforcement learning (RL), but their applicability remains largely confined to continuous action spaces. To address a broader range of offline RL settings, we extend flow matching to a general fra…

• F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare

  Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov, Viacheslav Sinii, Alexey Malakhov, Daniil Gavrilov · 9. Februar 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) is commonly based on group sampling to estimate advantages and stabilize policy updates. In practice, large group sizes are not feasible due to computational limits, which biases learning toward trajectories that are already likely. Smaller group…

• Difficulty-Estimated Policy Optimization

  Yu Zhao, Fan Jiang, Tianle Liu, Bo Zeng, Yu Liu, Longyue Wang, Weihua Luo · 9. Februar 2026

  Recent advancements in Large Reasoning Models (LRMs), exemplified by DeepSeek-R1, have underscored the potential of scaling inference-time compute through Group Relative Policy Optimization (GRPO). However, GRPO frequently suffers from gradient signal attenuation when encountering problems that are …

• Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities

  Marco Bagatella, Thomas Rupf, Georg Martius, Andreas Krause · 9. Februar 2026

  Recent advancements in zero-shot reinforcement learning (RL) have facilitated the extraction of diverse behaviors from unlabeled, offline data sources. In particular, forward-backward algorithms (FB) can retrieve a family of policies that can approximately solve any standard RL problem (with additiv…

• F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare

  Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov, Viacheslav Sinii, Alexey Malakhov, Daniil Gavrilov · 9. Februar 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) is commonly based on group sampling to estimate advantages and stabilize policy updates. In practice, large group sizes are not feasible due to computational limits, which biases learning toward trajectories that are already likely. Smaller group…

• Prism: Spectral Parameter Sharing for Multi-Agent Reinforcement Learning

  Kyungbeom Kim, Seungwon Oh, Kyung-Joong Kim · 9. Februar 2026

  Parameter sharing is a key strategy in multi-agent reinforcement learning (MARL) for improving scalability, yet conventional fully shared architectures often collapse into homogeneous behaviors. Recent methods introduce diversity through clustering, pruning, or masking, but typically compromise reso…

• Progress Constraints for Reinforcement Learning in Behavior Trees

  Finn Rietz, Mart Karta\v{s}ev, Johannes A. Stork, Petter \"Ogren · 9. Februar 2026

  Behavior Trees (BTs) provide a structured and reactive framework for decision-making, commonly used to switch between sub-controllers based on environmental conditions. Reinforcement Learning (RL), on the other hand, can learn near-optimal controllers but sometimes struggles with sparse rewards, saf…

• Sample-Efficient Policy Space Response Oracles with Joint Experience Best Response

  Ariyan Bighashdel, Thiago D. Sim\~ao, Frans A. Oliehoek · 9. Februar 2026

  Multi-agent reinforcement learning (MARL) offers a scalable alternative to exact game-theoretic analysis but suffers from non-stationarity and the need to maintain diverse populations of strategies that capture non-transitive interactions. Policy Space Response Oracles (PSRO) address these issues by…

• ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training

  Dunwei Tu, Hongyan Hao, Hansi Yang, Yihao Chen, Yi-Kai Zhang, Zhikang Xia, Yu Yang, Yueqing Sun, Xingchen Liu, Furao Shen, Qi Gu, Hui Su, Xunliang Cai · 9. Februar 2026

  Training generalist agents capable of adapting to diverse scenarios requires interactive environments for self-exploration. However, interactive environments remain critically scarce, and existing synthesis methods suffer from significant limitations regarding environmental diversity and scalability…

• SeeUPO: Sequence-Level Agentic-RL with Convergence Guarantees

  Tianyi Hu, Qingxu Fu, Yanxi Chen, Zhaoyang Liu, Bolin Ding · 9. Februar 2026

  Reinforcement learning (RL) has emerged as the predominant paradigm for training large language model (LLM)-based AI agents. However, existing backbone RL algorithms lack verified convergence guarantees in agentic scenarios, especially in multi-turn settings, which can lead to training instability a…

• Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch Reinforcement Learning

  Zhuoming Chen, Hongyi Liu, Yang Zhou, Haizhong Zheng, Beidi Chen · 9. Februar 2026

  Reinforcement learning (RL) for large language models (LLMs) remains expensive, particularly because the rollout is expensive. Decoupling rollout generation from policy optimization (e.g., leveraging a more efficient model to rollout) could enable substantial efficiency gains, yet doing so introduce…

• Continuous-time reinforcement learning: ellipticity enables model-free value function approximation

  Wenlong Mou · 9. Februar 2026

  We study off-policy reinforcement learning for controlling continuous-time Markov diffusion processes with discrete-time observations and actions. We consider model-free algorithms with function approximation that learn value and advantage functions directly from data, without unrealistic structural…

• Cochain Perspectives on Temporal-Difference Signals for Learning Beyond Markov Dynamics

  Zuyuan Zhang, Sizhe Tang, Tian Lan · 9. Februar 2026

  Non-Markovian dynamics are commonly found in real-world environments due to long-range dependencies, partial observability, and memory effects. The Bellman equation that is the central pillar of Reinforcement learning (RL) becomes only approximately valid under Non-Markovian. Existing work often foc…

• When Are RL Hyperparameters Benign? A Study in Offline Goal-Conditioned RL

  Jan Malte T\"opperwien, Aditya Mohan, Marius Lindauer · 6. Februar 2026

  Hyperparameter sensitivity in Deep Reinforcement Learning (RL) is often accepted as unavoidable. However, it remains unclear whether it is intrinsic to the RL problem or exacerbated by specific training mechanisms. We investigate this question in offline goal-conditioned RL, where data distributions…

• CARL: Focusing Agentic Reinforcement Learning on Critical Actions

  Leyang Shen, Yang Zhang, Chun Kai Ling, Xiaoyan Zhao, Tat-Seng Chua · 6. Februar 2026

  Agents capable of accomplishing complex tasks through multiple interactions with the environment have emerged as a popular research direction. However, in such multi-step settings, the conventional group-level policy optimization algorithm becomes suboptimal because of its underlying assumption that…

• Rewards as Labels: Revisiting RLVR from a Classification Perspective

  Zepeng Zhai, Meilin Chen, Jiaxuan Zhao, Junlang Qian, Lei Shen, Yuan Lu · 6. Februar 2026

  Reinforcement Learning with Verifiable Rewards has recently advanced the capabilities of Large Language Models in complex reasoning tasks by providing explicit rule-based supervision. Among RLVR methods, GRPO and its variants have achieved strong empirical performance. Despite their success, we iden…

• Privileged Information Distillation for Language Models

  Emiliano Penaloza, Dheeraj Vattikonda, Nicolas Gontier, Alexandre Lacoste, Laurent Charlin, Massimo Caccia · 6. Februar 2026

  Training-time privileged information (PI) can enable language models to succeed on tasks they would otherwise fail, making it a powerful tool for reinforcement learning in hard, long-horizon settings. However, transferring capabilities learned with PI to policies that must act without it at inferenc…

• ReFORM: Reflected Flows for On-support Offline RL via Noise Manipulation

  Songyuan Zhang, Oswin So, H. M. Sabbir Ahmad, Eric Yang Yu, Matthew Cleaveland, Mitchell Black, Chuchu Fan · 6. Februar 2026

  Offline reinforcement learning (RL) aims to learn the optimal policy from a fixed dataset generated by behavior policies without additional environment interactions. One common challenge that arises in this setting is the out-of-distribution (OOD) error, which occurs when the policy leaves the train…

• Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access

  Daniel Ebi, Gaspard Lambrechts, Damien Ernst, Klemens B\"ohm · 6. Februar 2026

  Asymmetric actor-critic methods are widely used in partially observable reinforcement learning, but typically assume full state observability to condition the critic during training, which is often unrealistic in practice. We introduce the informed asymmetric actor-critic framework, allowing the cri…

• Perception-Based Beliefs for POMDPs with Visual Observations

  Miriam Sch\"afers, Merlijn Krale, Thiago D. Sim\~ao, Nils Jansen, Maximilian Weininger · 6. Februar 2026

  Partially observable Markov decision processes (POMDPs) are a principled planning model for sequential decision-making under uncertainty. Yet, real-world problems with high-dimensional observations, such as camera images, remain intractable for traditional belief- and filtering-based solvers. To tac…

• DFPO: Scaling Value Modeling via Distributional Flow towards Robust and Generalizable LLM Post-Training

  Dingwei Zhu, Zhiheng Xi, Shihan Dou, Jiahan Li, Chenhao Huang, Junjie Ye, Sixian Li, Mingxu Chai, Yuhui Wang, Yajie Yang, Ming Zhang, Jiazheng Zhang, Shichun Liu, Caishuang Huang, Yunke Zhang, Yuran Wang, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang · 6. Februar 2026

  Training reinforcement learning (RL) systems in real-world environments remains challenging due to noisy supervision and poor out-of-domain (OOD) generalization, especially in LLM post-training. Recent distributional RL methods improve robustness by modeling values with multiple quantile points, but…

• Anchored Policy Optimization: Mitigating Exploration Collapse Via Support-Constrained Rectification

  Tianyi Wang, Long Li, Hongcan Guo, Yibiao Chen, Yixia Li, Yong Wang, Yun Chen, Guanhua Chen · 6. Februar 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) is increasingly viewed as a tree pruning mechanism. However, we identify a systemic pathology termed Recursive Space Contraction (RSC), an irreversible collapse driven by the combined dynamics of positive sharpening and negative squeezing, where …

• $f$-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment

  Rajdeep Haldar, Lantao Mei, Guang Lin, Yue Xing, Qifan Song · 6. Februar 2026

  Recent research shows that Preference Alignment (PA) objectives act as divergence estimators between aligned (chosen) and unaligned (rejected) response distributions. In this work, we extend this divergence-based perspective to general alignment settings, such as reinforcement learning with verifiab…