Reinforcement Learning in Robotics

Latest papers

• The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs

  Jingdi Chen, Hanqing Yang, Zongjun Liu, Carlee Joe-Wong · 13 February 2026

  Multi-agent sequential decision-making powers many real-world systems, from autonomous vehicles and robotics to collaborative AI assistants. In dynamic, partially observable environments, communication is often what reduces uncertainty and makes collaboration possible. This survey reviews multi-agen…

• Adaptive Milestone Reward for GUI Agents

  Congmin Zheng, Xiaoyun Mo, Xinbei Ma, Qiqiang Lin, Yin Zhao, Jiachen Zhu, Xingyu Lou, Jun Wang, Zhaoxiang Wang, Weiwen Liu, Zhuosheng Zhang, Yong Yu, Weinan Zhang · 13 February 2026

  Reinforcement Learning (RL) has emerged as a mainstream paradigm for training Mobile GUI Agents, yet it struggles with the temporal credit assignment problem inherent in long-horizon tasks. A primary challenge lies in the trade-off between reward fidelity and density: outcome reward offers high fide…

• Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training

  Miaosen Zhang, Yishan Liu, Shuxia Lin, Xu Yang, Qi Dai, Chong Luo, Weihao Jiang, Peng Hou, Anxiang Zeng, Xin Geng, Baining Guo · 13 February 2026

  Supervised fine-tuning (SFT) is computationally efficient but often yields inferior generalization compared to reinforcement learning (RL). This gap is primarily driven by RL's use of on-policy data. We propose a framework to bridge this chasm by enabling On-Policy SFT. We first present \textbf{\tex…

• CM2: Reinforcement Learning with Checklist Rewards for Multi-Turn and Multi-Step Agentic Tool Use

  Zhen Zhang, Kaiqiang Song, Xun Wang, Yebowen Hu, Weixiang Yan, Chenyang Zhao, Henry Peng Zou, Haoyun Deng, Sathish Reddy Indurthi, Shujian Liu, Simin Ma, Xiaoyang Wang, Xin Eric Wang, Song Wang · 13 February 2026

  AI agents are increasingly used to solve real-world tasks by reasoning over multi-turn user interactions and invoking external tools. However, applying reinforcement learning to such settings remains difficult: realistic objectives often lack verifiable rewards and instead emphasize open-ended behav…

• Temporal Difference Learning with Constrained Initial Representations

  Jiafei Lyu, Jingwen Yang, Zhongjian Qiao, Runze Liu, Zeyuan Liu, Deheng Ye, Zongqing Lu, Xiu Li · 13 February 2026

  Recently, there have been numerous attempts to enhance the sample efficiency of off-policy reinforcement learning (RL) agents when interacting with the environment, including architecture improvements and new algorithms. Despite these advances, they overlook the potential of directly constraining th…

• Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization

  Chengrui Qu, Christopher Yeh, Kishan Panaganti, Eric Mazumdar, Adam Wierman · 13 February 2026

  Cooperative multi-agent reinforcement learning (MARL) commonly adopts centralized training with decentralized execution, where value-factorization methods enforce the individual-global-maximum (IGM) principle so that decentralized greedy actions recover the team-optimal joint action. However, the re…

• Provable Offline Reinforcement Learning for Structured Cyclic MDPs

  Kyungbok Lee, Angelica Cristello Sarteau, Michael R. Kosorok · 13 February 2026

  We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is challenging: optimizing a policy at any stage shifts the state dis…

• Can We Really Learn One Representation to Optimize All Rewards?

  Chongyi Zheng, Royina Karegoudra Jayanth, Benjamin Eysenbach · 13 February 2026

  As machine learning has moved towards leveraging large models as priors for downstream tasks, the community has debated the right form of prior for solving reinforcement learning (RL) problems. If one were to try to prefetch as much computation as possible, they would attempt to learn a prior over t…

• Geometry of Uncertainty: Learning Metric Spaces for Multimodal State Estimation in RL

  Alfredo Reichlin, Adriano Pacciarelli, Danica Kragic, Miguel Vasco · 13 February 2026

  Estimating the state of an environment from high-dimensional, multimodal, and noisy observations is a fundamental challenge in reinforcement learning (RL). Traditional approaches rely on probabilistic models to account for the uncertainty, but often require explicit noise assumptions, in turn limiti…

• Unifying Stable Optimization and Reference Regularization in RLHF

  Li He, Qiang Qu, He Zhao, Stephen Wan, Dadong Wang, Lina Yao, Tongliang Liu · 13 February 2026

  Reinforcement Learning from Human Feedback (RLHF) has advanced alignment capabilities significantly but remains hindered by two core challenges: \textbf{reward hacking} and \textbf{stable optimization}. Current solutions independently address these issues through separate regularization strategies, …

• On the optimization dynamics of RLVR: Gradient gap and step size thresholds

  Joe Suk, Yaqi Duan · 13 February 2026

  Reinforcement Learning with Verifiable Rewards (RLVR), which uses simple binary feedback to post-train large language models, has found significant empirical success. However, a principled understanding of why it works is lacking. This paper builds a theoretical foundation for RLVR by analyzing its …

• A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation

  Xinjie Liu, Cyrus Neary, Kushagra Gupta, Wesley A. Suttle, Christian Ellis, Ufuk Topcu, David Fridovich-Keil · 13 February 2026

  Many reinforcement learning (RL) algorithms are impractical for training in operational systems or computationally expensive high-fidelity simulations, as they require large amounts of data. Meanwhile, low-fidelity simulators, e.g., reduced-order models, heuristic rewards, or learned world models, c…

• Provably Efficient Algorithms for S- and Non-Rectangular Robust MDPs with General Parameterization

  Anirudh Satheesh, Ziyi Chen, Furong Huang, Heng Huang · 13 February 2026

  We study robust Markov decision processes (RMDPs) with general policy parameterization under s-rectangular and non-rectangular uncertainty sets. Prior work is largely limited to tabular policies, and hence either lacks sample complexity guarantees or incurs high computational cost. Our method reduce…

• TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents

  Aladin Djuhera, Swanand Ravindra Kadhe, Farhan Ahmed, Holger Boche · 13 February 2026

  Advances in large language models (LLMs) are driving a shift toward using reinforcement learning (RL) to train agents from iterative, multi-turn interactions across tasks. However, multi-turn RL remains challenging as rewards are often sparse or delayed, and environments can be stochastic. In this r…

• Intrinsic-Energy Joint Embedding Predictive Architectures Induce Quasimetric Spaces

  Anthony Kobanda, Waris Radji · 13 February 2026

  Joint-Embedding Predictive Architectures (JEPAs) aim to learn representations by predicting target embeddings from context embeddings, inducing a scalar compatibility energy in a latent space. In contrast, Quasimetric Reinforcement Learning (QRL) studies goal-conditioned control through directed dis…

• Data-Efficient Hierarchical Goal-Conditioned Reinforcement Learning via Normalizing Flows

  Shaswat Garg, Matin Moezzi, Brandon Da Silva · 12 February 2026

  Hierarchical goal-conditioned reinforcement learning (H-GCRL) provides a powerful framework for tackling complex, long-horizon tasks by decomposing them into structured subgoals. However, its practical adoption is hindered by poor data efficiency and limited policy expressivity, especially in offlin…

• General Flexible $f$-divergence for Challenging Offline RL Datasets with Low Stochasticity and Diverse Behavior Policies

  Jianxun Wang, Grant C. Forbes, Leonardo Villalobos-Arias, David L. Roberts · 12 February 2026

  Offline RL algorithms aim to improve upon the behavior policy that produces the collected data while constraining the learned policy to be within the support of the dataset. However, practical offline datasets often contain examples with little diversity or limited exploration of the environment, an…

• Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization

  Simon Sinong Zhan, Qingyuan Wu, Philip Wang, Frank Yang, Xiangyu Shi, Chao Huang, Qi Zhu · 12 February 2026

  Offline-to-online deployment of reinforcement-learning (RL) agents must bridge two gaps: (1) the sim-to-real gap, where real systems add latency and other imperfections not present in simulation, and (2) the interaction gap, where policies trained purely offline face out-of-distribution states durin…

• VESPO: Variational Sequence-Level Soft Policy Optimization for Stable Off-Policy LLM Training

  Guobin Shen, Chenxiao Zhao, Xiang Cheng, Lei Huang, Xing Yu · 12 February 2026

  Training stability remains a central challenge in reinforcement learning (RL) for large language models (LLMs). Policy staleness, asynchronous training, and mismatches between training and inference engines all cause the behavior policy to diverge from the current policy, risking training collapse. …

• Confounding Robust Continuous Control via Automatic Reward Shaping

  Mateo Juliani, Mingxuan Li, Elias Bareinboim · 12 February 2026

  Reward shaping has been applied widely to accelerate Reinforcement Learning (RL) agents' training. However, a principled way of designing effective reward shaping functions, especially for complex continuous control problems, remains largely under-explained. In this work, we propose to automatically…

• OSIL: Learning Offline Safe Imitation Policies with Safety Inferred from Non-preferred Trajectories

  Returaj Burnwal, Nirav Pravinbhai Bhatt, Balaraman Ravindran · 12 February 2026

  This work addresses the problem of offline safe imitation learning (IL), where the goal is to learn safe and reward-maximizing policies from demonstrations that do not have per-timestep safety cost or reward information. In many real-world domains, online learning in the environment can be risky, an…

• What Makes Value Learning Efficient in Residual Reinforcement Learning?

  Guozheng Ma, Lu Li, Haoyu Wang, Zixuan Liu, Pierre-Luc Bacon, Dacheng Tao · 12 February 2026

  Residual reinforcement learning (RL) enables stable online refinement of expressive pretrained policies by freezing the base and learning only bounded corrections. However, value learning in residual RL poses unique challenges that remain poorly understood. In this work, we identify two key bottlene…

• OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization

  Keane Ong, Sabri Boughorbel, Luwei Xiao, Chanakya Ekbote, Wei Dai, Ao Qu, Jingyao Wu, Rui Mao, Ehsan Hoque, Erik Cambria, Gianmarco Mengaldo, Paul Pu Liang · 12 February 2026

  To develop socially intelligent AI, existing approaches typically model human behavioral dimensions (e.g., affective, cognitive, or social attributes) in isolation. Although useful, task-specific modeling often increases training costs and limits generalization across behavioral settings. Recent rea…

• Metareasoning in uncertain environments: a meta-BAMDP framework

  Prakhar Godara, Tilman Diego Alem\'an · 12 February 2026

  \textit{Reasoning} may be viewed as an algorithm $P$ that makes a choice of an action $a^* \in \mathcal{A}$, aiming to optimize some outcome. However, executing $P$ itself bears costs (time, energy, limited capacity, etc.) and needs to be considered alongside explicit utility obtained by making the …

• Neuro-symbolic Action Masking for Deep Reinforcement Learning

  Shuai Han, Mehdi Dastani, Shihan Wang · 12 February 2026

  Deep reinforcement learning (DRL) may explore infeasible actions during training and execution. Existing approaches assume a symbol grounding function that maps high-dimensional states to consistent symbolic representations and a manually specified action masking techniques to constrain actions. In …