Reinforcement Learning in Robotics

Neueste Paper

• ARM: Advantage Reward Modeling for Long-Horizon Manipulation

  Yiming Mao, Zixi Yu, Weixin Mao, Yinhao Li, Qirui Hu, Zihan Lan, Minzhao Zhu, Hua Chen · 6. April 2026

  Long-horizon robotic manipulation remains challenging for reinforcement learning (RL) because sparse rewards provide limited guidance for credit assignment. Practical policy improvement thus relies on richer intermediate supervision, such as dense progress rewards, which are costly to obtain and ill…

• Contextual Intelligence The Next Leap for Reinforcement Learning

  Andr\'e Biedenkapp · 6. April 2026

  Reinforcement learning (RL) has produced spectacular results in games, robotics, and continuous control. Yet, despite these successes, learned policies often fail to generalize beyond their training distribution, limiting real-world impact. Recent work on contextual RL (cRL) shows that exposing agen…

• Prism: Policy Reuse via Interpretable Strategy Mapping in Reinforcement Learning

  Thomas Pravetz · 6. April 2026

  We present PRISM (Policy Reuse via Interpretable Strategy Mapping), a framework that grounds reinforcement learning agents' decisions in discrete, causally validated concepts and uses those concepts as a zero-shot transfer interface between agents trained with different algorithms. PRISM clusters ea…

• Coupled Control, Structured Memory, and Verifiable Action in Agentic AI (SCRAT -- Stochastic Control with Retrieval and Auditable Trajectories): A Comparative Perspective from Squirrel Locomotion and Scatter-Hoarding

  Maximiliano Armesto, Christophe Kolb · 6. April 2026

  Agentic AI is increasingly judged not by fluent output alone but by whether it can act, remember, and verify under partial observability, delay, and strategic observation. Existing research often studies these demands separately: robotics emphasizes control, retrieval systems emphasize memory, and a…

• Beyond Noisy-TVs: Noise-Robust Exploration Via Learning Progress Monitoring

  Zhibo Hou, Zhiyu An, Wan Du · 6. April 2026

  When there exists an unlearnable source of randomness (noisy-TV) in the environment, a naively intrinsic reward driven exploring agent gets stuck at that source of randomness and fails at exploration. Intrinsic reward based on uncertainty estimation or distribution similarity, while eventually escap…

• Self-Distilled RLVR

  Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, Nan Duan · 6. April 2026

  On-policy distillation (OPD) has become a popular training paradigm in the LLM community. This paradigm selects a larger model as the teacher to provide dense, fine-grained signals for each sampled trajectory, in contrast to reinforcement learning with verifiable rewards (RLVR), which only obtains s…

• Hierarchical Planning with Latent World Models

  Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, Nicolas Ballas · 6. April 2026

  Model predictive control (MPC) with learned world models has emerged as a promising paradigm for embodied control, particularly for its ability to generalize zero-shot when deployed in new environments. However, learned world models often struggle with long-horizon control due to the accumulation of…

• OPRIDE: Offline Preference-based Reinforcement Learning via In-Dataset Exploration

  Yiqin Yang, Hao Hu, Yihuan Mao, Jin Zhang, Chengjie Wu, Yuhua Jiang, Xu Yang, Runpeng Xie, Yi Fan, Bo Liu, Yang Gao, Bo Xu, Chongjie Zhang · 6. April 2026

  Preference-based reinforcement learning (PbRL) can help avoid sophisticated reward designs and align better with human intentions, showing great promise in various real-world applications. However, obtaining human feedback for preferences can be expensive and time-consuming, which forms a strong bar…

• Reflective Context Learning: Studying the Optimization Primitives of Context Space

  Nikita Vassilyev, William Berrios, Ruowang Zhang, Bo Han, Douwe Kiela, Shikib Mehri · 6. April 2026

  Generally capable agents must learn from experience in ways that generalize across tasks and environments. The fundamental problems of learning, including credit assignment, overfitting, forgetting, local optima, and high-variance learning signals, persist whether the learned object lies in paramete…

• Mitigating Reward Hacking in RLHF via Advantage Sign Robustness

  Shinnosuke Ono, Johannes Ackermann, Soichiro Nishimori, Takashi Ishida, Masashi Sugiyama · 6. April 2026

  Reward models (RMs) used in reinforcement learning from human feedback (RLHF) are vulnerable to reward hacking: as the policy maximizes a learned proxy reward, true quality plateaus or degrades. We make the assumption that reward hacking is often caused by flipped advantage signs: instead of reducin…

• Malliavin Calculus for Counterfactual Gradient Estimation in Adaptive Inverse Reinforcement Learning

  Vikram Krishnamurthy, Luke Snow · 3. April 2026

  Inverse reinforcement learning (IRL) recovers the loss function of a forward learner from its observed responses adaptive IRL aims to reconstruct the loss function of a forward learner by passively observing its gradients as it performs reinforcement learning (RL). This paper proposes a novel passiv…

• Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Error

  Taisuke Kobayashi · 3. April 2026

  In reinforcement learning (RL), temporal difference (TD) errors are widely adopted for optimizing value and policy functions. However, since the TD error is defined by a bootstrap method, its computation tends to be noisy and destabilize learning. Heuristics to improve the accuracy of TD errors, suc…

• Learning from the Right Rollouts: Data Attribution for PPO-based LLM Post-Training

  Dong Shu, Denghui Zhang, Jessica Hullman · 3. April 2026

  Traditional RL algorithms like Proximal Policy Optimization (PPO) typically train on the entire rollout buffer, operating under the assumption that all generated episodes provide a beneficial optimization signal. However, these episodes frequently contain noisy or unfaithful reasoning, which can deg…

• Model-Based Reinforcement Learning for Control under Time-Varying Dynamics

  Klemens Iten, Bruce Lee, Chenhao Li, Lenart Treven, Andreas Krause, Bhavya Sukhija · 3. April 2026

  Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcem…

• Constraint-Aware Reinforcement Learning via Adaptive Action Scaling

  Murad Dawood, Usama Ahmed Siddiquie, Shahram Khorshidi, Maren Bennewitz · 3. April 2026

  Safe reinforcement learning (RL) seeks to mitigate unsafe behaviors that arise from exploration during training by reducing constraint violations while maintaining task performance. Existing approaches typically rely on a single policy to jointly optimize reward and safety, which can cause instabili…

• SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

  Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen · 3. April 2026

  Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidanc…

• Monotone and Conservative Policy Iteration Beyond the Tabular Case

  S. R. Eshwar, Gugan Thoppe, Ananyabrata Barua, Aditya Gopalan, Gal Dalal · 3. April 2026

  We introduce Reliable Policy Iteration (RPI) and Conservative RPI (CRPI), variants of Policy Iteration (PI) and Conservative PI (CPI), that retain tabular guarantees under function approximation. RPI uses a novel Bellman-constrained optimization for policy evaluation. We show that RPI restores the t…

• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

  Yuejiang Liu, Fan Feng, Lingjing Kong, Weifeng Lu, Jinzhou Tang, Kun Zhang, Kevin Murphy, Chelsea Finn, Yilun Du · 3. April 2026

  General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning, which primarily focuses on optimal actions, a world model must be reliable over a much broader range of suboptimal …

• Residuals-based Offline Reinforcement Learning

  Qing Zhu, Xian Yu · 3. April 2026

  Offline reinforcement learning (RL) has received increasing attention for learning policies from previously collected data without interaction with the real environment, which is particularly important in high-stakes applications. While a growing body of work has developed offline RL algorithms, the…

• Computing the Exact Pareto Front in Average-Cost Multi-Objective Markov Decision Processes

  Jiping Luo, Nikolaos Pappas · 3. April 2026

  Many communication and control problems are cast as multi-objective Markov decision processes (MOMDPs). The complete solution to an MOMDP is the Pareto front. Much of the literature approximates this front via scalarization into single-objective MDPs. Recent work has begun to characterize the full f…

• Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

  Rafael Pardinas, Ehsan Kamalloo, David Vazquez, Alexandre Drouin · 3. April 2026

  Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely adopted by frontier open-weight models. However, their training recipes and domain mixtures are often not disclosed. Joint optimization across domains poses si…

• D-SPEAR: Dual-Stream Prioritized Experience Adaptive Replay for Stable Reinforcement Learning in Robotic Manipulation

  Yu Zhang, Karl Mason · 3. April 2026

  Robotic manipulation remains challenging for reinforcement learning due to contact-rich dynamics, long horizons, and training instability. Although off-policy actor-critic algorithms such as SAC and TD3 perform well in simulation, they often suffer from policy oscillations and performance collapse i…

• Soft MPCritic: Amortized Model Predictive Value Iteration

  Thomas Banker, Nathan P. Lawrence, Ali Mesbah · 3. April 2026

  Reinforcement learning (RL) and model predictive control (MPC) offer complementary strengths, yet combining them at scale remains computationally challenging. We propose soft MPCritic, an RL-MPC framework that learns in (soft) value space while using sample-based planning for both online control and…

• Thinking While Listening: Fast-Slow Recurrence for Long-Horizon Sequential Modeling

  Shota Takashiro, Masanori Koyama, Takeru Miyato, Yusuke Iwasawa, Yutaka Matsuo, Kohei Hayashi · 3. April 2026

  We extend the recent latent recurrent modeling to sequential input streams. By interleaving fast, recurrent latent updates with self-organizational ability between slow observation updates, our method facilitates the learning of stable internal structures that evolve alongside the input. This mechan…

• Auction-Based Online Policy Adaptation for Evolving Objectives

  Guruprerana Shabadi, Kaushik Mallik · 3. April 2026

  We consider multi-objective reinforcement learning problems where objectives come from an identical family -- such as the class of reachability objectives -- and may appear or disappear at runtime. Our goal is to design adaptive policies that can efficiently adjust their behaviors as the set of acti…