Reinforcement Learning in Robotics

Últimos artículos

• UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models

  Jiaqi Wang (Beijing University of Posts and Telecommunications, Beijing Academy of Artificial Intelligence), Haoge Deng (Beijing Academy of Artificial Intelligence), Ting Pan (Beijing Academy of Artificial Intelligence), Yang Liu (Beijing Academy of Artificial Intelligence), Chengyuan Wang (Beijing Academy of Artificial Intelligence), Fan Zhang (Beijing Academy of Artificial Intelligence), Yonggang Qi (Beijing University of Posts and Telecommunications), Xinlong Wang (Beijing Academy of Artificial Intelligence) · 21 de abril de 2026

  Uniform Discrete Diffusion Model (UDM) has recently emerged as a promising paradigm for discrete generative modeling; however, its integration with reinforcement learning remains largely unexplored. We observe that naively applying GRPO to UDM leads to training instability and marginal performance g…

• Does "Do Differentiable Simulators Give Better Policy Gradients?'' Give Better Policy Gradients?

  Ku Onoda, Paavo Parmas, Manato Yaguchi, Yutaka Matsuo · 21 de abril de 2026

  In policy gradient reinforcement learning, access to a differentiable model enables 1st-order gradient estimation that accelerates learning compared to relying solely on derivative-free 0th-order estimators. However, discontinuous dynamics cause bias and undermine the effectiveness of 1st-order esti…

• Bounded Ratio Reinforcement Learning

  Yunke Ao, Le Chen, Bruce D. Lee, Assefa S. Wahd, Aline Czarnobai, Philipp F\"urnstahl, Bernhard Sch\"olkopf, Andreas Krause · 21 de abril de 2026

  Proximal Policy Optimization (PPO) has become the predominant algorithm for on-policy reinforcement learning due to its scalability and empirical robustness across domains. However, there is a significant disconnect between the underlying foundations of trust region methods and the heuristic clipped…

• Physics-Informed Causal MDPs for Sequential Constraint Repair in Engineering Simulation Pipelines

  Chuhan Qiao · 21 de abril de 2026

  Off-policy learning in constrained MDPs with large binary state spaces faces a fundamental tension: causal identification of transition dynamics requires structural assumptions, while sample-efficient policy learning requires state-space compression. We introduce PI-CMDP, a framework for CMDPs whose…

• DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty

  Mingxuan Cui, Duo Zhou, Yuxuan Han, Grani A. Hanasusanto, Qiong Wang, Huan Zhang, Zhengyuan Zhou · 21 de abril de 2026

  Deep reinforcement learning (RL) has achieved remarkable success, yet its deployment in real-world scenarios is often limited by vulnerability to environmental uncertainties. Distributionally robust RL (DR-RL) algorithms have been proposed to resolve this challenge, but existing approaches are large…

• Inter-Agent Relative Representations for Multi-Agent Option Discovery

  Raul D. Steleac, Mohan Sridharan, David Abel · 21 de abril de 2026

  Temporally extended actions improve the ability to explore and plan in single-agent settings. In multi-agent settings, the exponential growth of the joint state space with the number of agents makes coordinated behaviours even more valuable. Yet, this same exponential growth renders the design of mu…

• Scale-free adaptive planning for deterministic dynamics & discounted rewards

  Peter L. Bartlett, Victor Gabillon, Jennifer Healey, Michal Valko · 21 de abril de 2026

  We address the problem of planning in an environment with deterministic dynamics and stochastic rewards with discounted returns. The optimal value function is not known, nor are the rewards bounded. We propose Platypoos, a simple scale-free planning algorithm that adapts to the unknown scale and smo…

• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data

  Zhenwen Liang, Yujun Zhou, Sidi Lu, Xiangliang Zhang, Haitao Mi, Dong Yu · 21 de abril de 2026

  Reinforcement Learning (RL) enhances LLM reasoning, yet a paradox emerges as models scale: strong base models saturate standard benchmarks (e.g., MATH), yielding correct but homogeneous solutions. In such environments, the lack of failure cases causes the advantage signal in group-relative algorithm…

• Learning-Based Sparsification of Dynamic Graphs in Robotic Exploration Algorithms

  Adithya V. Sastry, Bibek Poudel, Weizi Li · 21 de abril de 2026

  Many robotic exploration algorithms rely on graph structures for frontier-based exploration and dynamic path planning. However, these graphs grow rapidly, accumulating redundant information and impacting performance. We present a transformer-based framework trained with Proximal Policy Optimization …

• SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models

  Yifu Huo, Chenglong Wang, Ziming Zhu, Shunjie Xing, Peinan Feng, Tongran Liu, Qiaozhi He, Tianhua Zhou, Xiaojia Chang, Jingbo Zhu, Zhengtao Yu, Tong Xiao · 21 de abril de 2026

  Reinforcement learning (RL) has emerged as a promising paradigm for training reasoning-oriented models by leveraging rule-based reward signals. However, RL training typically tends to improve single-sample success rates (i.e., Pass@1) while offering limited exploration of diverse reasoning trajector…

• Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning

  Daniel Lawson, Adriana Hugessen, Charlotte Cloutier, Glen Berseth, Khimya Khetarpal · 21 de abril de 2026

  While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require conditioning on novel state-goal pairs, i.e. combinatorial generalization. In part, this limitation can be attributed to a lac…

• Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought

  Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga, Samet Oymak · 21 de abril de 2026

  State-of-the-art reasoning models utilize long chain-of-thought (CoT) to solve increasingly complex problems using more test-time computation. In this work, we explore a long CoT setting where the model makes up to K successive attempts at solving a problem, in which each attempt is allowed to build…

• Efficient Federated RLHF via Zeroth-Order Policy Optimization

  Deyi Wang, Qining Zhang, Lei Ying · 21 de abril de 2026

  This paper considers reinforcement learning from human feedback in a federated learning setting with resource-constrained agents, such as edge devices. We propose an efficient federated RLHF algorithm, named Partitioned, Sign-based Stochastic Zeroth-order Policy Optimization (Par-S$^2$ZPO). The algo…

• Empowering Multi-Turn Tool-Integrated Agentic Reasoning with Group Turn Policy Optimization

  Yifeng Ding, Hung Le, Songyang Han, Kangrui Ruan, Zhenghui Jin, Varun Kumar, Zijian Wang, Anoop Deoras · 21 de abril de 2026

  Training Large Language Models (LLMs) for multi-turn Tool-Integrated Reasoning (TIR) - where models iteratively reason, generate code, and verify through execution - remains challenging for existing reinforcement learning (RL) approaches. Current RL methods, exemplified by Group Relative Policy Opti…

• SVL: Goal-Conditioned Reinforcement Learning as Survival Learning

  Franki Nguimatsia Tiofack, Fabian Schramm, Th\'eotime Le Hellard, Justin Carpentier · 21 de abril de 2026

  Standard approaches to goal-conditioned reinforcement learning (GCRL) that rely on temporal-difference learning can be unstable and sample-inefficient due to bootstrapping. While recent work has explored contrastive and supervised formulations to improve stability, we present a probabilistic alterna…

• From $\log \pi$ to $\pi$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight

  Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Chaowen Hu, Cong Qin, Zekai Shao, Binbin Zheng, Lu Pan, Ke Zeng · 21 de abril de 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed a leap in Large Language Model (LLM) reasoning, yet its optimization dynamics remain fragile. Standard algorithms like GRPO enforce stability via "hard clipping", which inadvertently stifles exploration by discarding gradients of to…

• The Global Neural World Model: Spatially Grounded Discrete Topologies for Action-Conditioned Planning

  Noureddine Kermiche · 21 de abril de 2026

  We present the Global Neural World Model (GNWM), a self-stabilizing framework that achieves topological quantization through balanced continuous entropy constraints. Operating as a continuous, action-conditioned Joint-Embedding Predictive Architecture (JEPA), the GNWM maps environments onto a discre…

• DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees

  Argyrios Gerogiannis, Yu-Han Huang, Venugopal V. Veeravalli · 21 de abril de 2026

  We study model-free reinforcement learning (RL) in non-stationary finite-horizon episodic Markov decision processes (MDPs) without prior knowledge of the non-stationarity. We focus on the piecewise-stationary (PS) setting, where both the reward and transition dynamics can change an arbitrary number …

• Positive-Only Drifting Policy Optimization

  Qi Zhang · 21 de abril de 2026

  In the field of online reinforcement learning (RL), traditional Gaussian policies and flow-based methods are often constrained by their unimodal expressiveness, complex gradient clipping, or stringent trust-region requirements. Moreover, they all rely on post-hoc penalization of negative samples to …

• Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample Construction

  Fei Ding, Yongkang Zhang, Runhao Liu, Yuhao Liao, Zijian Zeng, Huiming Yang, Sibo wang, Linglin Liao · 21 de abril de 2026

  This paper investigates the length problem in sequence-level relative reinforcement learning. We observe that, although existing methods partially alleviate length-related phenomena, a more fundamental issue remains insufficiently characterized: the comparison units used during training lack inheren…

• AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems

  Sumeet Ramesh Motwani, Chuan Du, Aleksander Petrov, Christopher Davis, Philip Torr, Antonio Papania-Davis, Weishi Yan · 21 de abril de 2026

  Optimization problems are central to decision-making in manufacturing, logistics, scheduling, and other industrial settings. Translating complicated descriptions of these problems into solver-ready formulations requires specialized operations research (OR) expertise, making it hard to scale. We pres…

• GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning

  Hikaru Shindo, Henri R\"o{\ss}ler, Quentin Delfosse, Kristian Kersting · 21 de abril de 2026

  Neuro-symbolic Reinforcement Learning (NeSy-RL) combines symbolic reasoning with gradient-based optimization to achieve interpretable and generalizable policies. Relational concepts, such as "left of" or "close by", serve as foundational building blocks that structure how agents perceive and act. Ho…

• Fisher Decorator: Refining Flow Policy via A Local Transport Map

  Xiaoyuan Cheng, Haoyu Wang, Wenxuan Yuan, Ziyan Wang, Zonghao Chen, Li Zeng, Zhuo Sun · 21 de abril de 2026

  Recent advances in flow-based offline reinforcement learning (RL) have achieved strong performance by parameterizing policies via flow matching. However, they still face critical trade-offs among expressiveness, optimality, and efficiency. In particular, existing flow policies interpret the $L_2$ re…

• Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning

  Lute Lillo, Nick Cheney · 20 de abril de 2026

  Continual reinforcement learning must balance retention with adaptation, yet many methods still rely on \emph{single-model preservation}, committing to one evolving policy as the main reusable solution across tasks. Even when a previously successful policy is retained, it may no longer provide a rel…

• EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems

  Yufei He, Juncheng Liu, Yue Liu, Yibo Li, Tri Cao, Zhiyuan Hu, Xinxing Xu, Bryan Hooi · 20 de abril de 2026

  A fundamental limitation of current AI agents is their inability to learn complex skills on the fly at test time, often behaving like "clever but clueless interns" in novel environments. This severely limits their practical utility. To systematically measure and drive progress on this challenge, we …