Reinforcement Learning in Robotics

Últimos artículos

• Online Risk-Averse Planning in POMDPs Using Iterated CVaR Value Function

  Yaacov Pariente, Vadim Indelman · 29 de enero de 2026

  We study risk-sensitive planning under partial observability using the dynamic risk measure Iterated Conditional Value-at-Risk (ICVaR). A policy evaluation algorithm for ICVaR is developed with finite-time performance guarantees that do not depend on the cardinality of the action space. Building on …

• In-Context Reinforcement Learning From Suboptimal Historical Data

  Juncheng Dong, Moyang Guo, Ethan X. Fang, Zhuoran Yang, Vahid Tarokh · 29 de enero de 2026

  Transformer models have achieved remarkable empirical successes, largely due to their in-context learning capabilities. Inspired by this, we explore training an autoregressive transformer for in-context reinforcement learning (ICRL). In this setting, we initially train a transformer on an offline da…

• Regularized Gradient Temporal-Difference Learning

  Hyunjun Na, Donghwan Lee · 29 de enero de 2026

  Gradient temporal-difference (GTD) learning algorithms are widely used for off-policy policy evaluation with function approximation. However, existing convergence analyses rely on the restrictive assumption that the so-called feature interaction matrix (FIM) is nonsingular. In practice, the FIM can …

• Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models

  Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Xiaobo Xia, Ming-Kun Xie, Dong-Dong Wu, Biao Liu, Yuheng Jia, Xin Geng, Masashi Sugiyama, Tat-Seng Chua · 29 de enero de 2026

  Due to constraints on privacy, cost, and latency, on-premise deployment of small models is increasingly common. However, most practical pipelines stop at supervised fine-tuning (SFT) and fail to reach the reinforcement learning (RL) alignment stage. The main reason is that RL alignment typically req…

• Deep SPI: Safe Policy Improvement via World Models

  Florent Delgrange, Raphael Avalos, Willem R\"opke · 29 de enero de 2026

  Safe policy improvement (SPI) offers theoretical control over policy updates, yet existing guarantees largely concern offline, tabular reinforcement learning (RL). We study SPI in general online settings, when combined with world model and representation learning. We develop a theoretical framework …

• GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning

  Zhiheng Jiang, Yunzhe Wang, Ryan Marr, Ellen Novoseller, Benjamin T. Files, Volkan Ustun · 29 de enero de 2026

  Preference-Conditioned Policy Learning (PCPL) in Multi-Objective Reinforcement Learning (MORL) aims to approximate diverse Pareto-optimal solutions by conditioning policies on user-specified preferences over objectives. This enables a single model to flexibly adapt to arbitrary trade-offs at run-tim…

• E2HiL: Entropy-Guided Sample Selection for Efficient Real-World Human-in-the-Loop Reinforcement Learning

  Haoyuan Deng, Yuanjiang Xue, Haoyang Du, Boyang Zhou, Zhenyu Wu, Ziwei Wang · 29 de enero de 2026

  Human-in-the-loop guidance has emerged as an effective approach for enabling faster convergence in online reinforcement learning (RL) of complex real-world manipulation tasks. However, existing human-in-the-loop RL (HiL-RL) frameworks often suffer from low sample efficiency, requiring substantial hu…

• Lifted Forward Planning in Relational Factored Markov Decision Processes with Concurrent Actions

  Florian Andreas Marwitz, Tanya Braun, Ralf M\"oller, Marcel Gehrke · 29 de enero de 2026

  Decision making is a central problem in AI that can be formalized using a Markov Decision Process. A problem is that, with increasing numbers of (indistinguishable) objects, the state space grows exponentially. To compute policies, the state space has to be enumerated. Even more possibilities have t…

• Distributional value gradients for stochastic environments

  Baptiste Debes, Tinne Tuytelaars · 29 de enero de 2026

  Gradient-regularized value learning methods improve sample efficiency by leveraging learned models of transition dynamics and rewards to estimate return gradients. However, existing approaches, such as MAGE, struggle in stochastic or noisy environments, limiting their applicability. In this work, we…

• Discrete Variational Autoencoding via Policy Search

  Michael Drolet, Firas Al-Hafez, Aditya Bhatt, Jan Peters, Oleg Arenz · 29 de enero de 2026

  Discrete latent bottlenecks in variational autoencoders (VAEs) offer high bit efficiency and can be modeled with autoregressive discrete distributions, enabling parameter-efficient multimodal search with transformers. However, discrete random variables do not allow for exact differentiable parameter…

• Less is More: Clustered Cross-Covariance Control for Offline RL

  Nan Qiao, Sheng Yue, Shuning Wang, Yongheng Deng, Ju Ren · 29 de enero de 2026

  A fundamental challenge in offline reinforcement learning is distributional shift. Scarce data or datasets dominated by out-of-distribution (OOD) areas exacerbate this issue. Our theoretical analysis and experiments show that the standard squared error objective induces a harmful TD cross covariance…

• Meta-Cognitive Reinforcement Learning with Self-Doubt and Recovery

  Zhipeng Zhang, Wenting Ma, Kai Li, Meng Guo, Lei Yang, Wei Yu, Hongji Cui, Yichen Zhang, Mo Zhang, Jinzhe Lin, Zhenjie Yao · 29 de enero de 2026

  Robust reinforcement learning methods typically focus on suppressing unreliable experiences or corrupted rewards, but they lack the ability to reason about the reliability of their own learning process. As a result, such methods often either overreact to noise by becoming overly conservative or fail…

• Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning

  Jinyang Wu, Shuo Yang, Changpeng Yang, Yuhao Shen, Shuai Zhang, Zhengqi Wen, Jianhua Tao · 29 de enero de 2026

  Reinforcement learning has empowered large language models to act as intelligent agents, yet training them for long-horizon tasks remains challenging due to the scarcity of high-quality trajectories, especially under limited resources. Existing methods typically scale up rollout sizes and indiscrimi…

• Neural Value Iteration

  Yang You, Ufuk \c{C}ak{\i}r, Alex Schutz, Nick Hawes · 29 de enero de 2026

  The value function of a POMDP exhibits the piecewise-linear-convex (PWLC) property and can be represented as a finite set of hyperplanes, known as $\alpha$-vectors. Most state-of-the-art POMDP solvers (offline planners) follow the point-based value iteration scheme, which performs Bellman backups on…

• From Observations to Events: Event-Aware World Model for Reinforcement Learning

  Zhao-Han Peng, Shaohui Li, Zhi Li, Shulan Ruan, Yu Liu, You He · 28 de enero de 2026

  While model-based reinforcement learning (MBRL) improves sample efficiency by learning world models from raw observations, existing methods struggle to generalize across structurally similar scenes and remain vulnerable to spurious variations such as textures or color shifts. From a cognitive scienc…

• APC-RL: Exceeding Data-Driven Behavior Priors with Adaptive Policy Composition

  Finn Rietz, Pedro Zuidberg dos Martires, Johannes Andreas Stork · 28 de enero de 2026

  Incorporating demonstration data into reinforcement learning (RL) can greatly accelerate learning, but existing approaches often assume demonstrations are optimal and fully aligned with the target task. In practice, demonstrations are frequently sparse, suboptimal, or misaligned, which can degrade p…

• Toward Learning POMDPs Beyond Full-Rank Actions and State Observability

  Seiji Shaw, Travis Manderson, Chad Kessens, Nicholas Roy · 28 de enero de 2026

  We are interested in enabling autonomous agents to learn and reason about systems with hidden states, such as furniture with hidden locking mechanisms. We cast this problem as learning the parameters of a discrete Partially Observable Markov Decision Process (POMDP). The agent begins with knowledge …

• A Unifying View of Coverage in Linear Off-Policy Evaluation

  Philip Amortila, Audrey Huang, Akshay Krishnamurthy, Nan Jiang · 28 de enero de 2026

  Off-policy evaluation (OPE) is a fundamental task in reinforcement learning (RL). In the classic setting of linear OPE, finite-sample guarantees often take the form $$ \textrm{Evaluation error} \le \textrm{poly}(C^\pi, d, 1/n,\log(1/\delta)), $$ where $d$ is the dimension of the features and $C^\pi$…

• Safe Exploration via Policy Priors

  Manuel Wendl, Yarden As, Manish Prajapat, Anton Pollak, Stelian Coros, Andreas Krause · 28 de enero de 2026

  Safe exploration is a key requirement for reinforcement learning (RL) agents to learn and adapt online, beyond controlled (e.g. simulated) environments. In this work, we tackle this challenge by utilizing suboptimal yet conservative policies (e.g., obtained from offline data or simulators) as priors…

• Risk-Sensitive Agent Compositions

  Guruprerana Shabadi, Rajeev Alur · 28 de enero de 2026

  From software development to robot control, modern agentic systems decompose complex objectives into a sequence of subtasks and choose a set of specialized AI agents to complete them. We formalize agentic workflows as directed acyclic graphs, called agent graphs, where edges represent AI agents and …

• Improving Policy Exploitation in Online Reinforcement Learning with Instant Retrospect Action

  Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia · 28 de enero de 2026

  Existing value-based online reinforcement learning (RL) algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Di…

• Scalable Exploration for High-Dimensional Continuous Control via Value-Guided Flow

  Yunyue Wei, Chenhui Zuo, Yanan Sui · 28 de enero de 2026

  Controlling high-dimensional systems in biological and robotic applications is challenging due to expansive state-action spaces, where effective exploration is critical. Commonly used exploration strategies in reinforcement learning are largely undirected with sharp degradation as action dimensional…

• Self-Distillation Enables Continual Learning

  Idan Shenfeld, Mehul Damani, Jonas H\"ubotter, Pulkit Agrawal · 28 de enero de 2026

  Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Lear…

• Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning

  Tongxi Wang, Zhuoyang Xia, Xinran Chen, Shan Liu · 28 de enero de 2026

  Real-world reinforcement learning often faces environment drift, but most existing methods rely on static entropy coefficients/target entropy, causing over-exploration during stable periods and under-exploration after drift (thus slow recovery), and leaving unanswered the principled question of how …

• R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning

  Zhizheng Jiang, Kang Zhao, Weikai Xu, Xinkui Lin, Wei Liu, Jian Luan, Shuo Shang, Peng Han · 28 de enero de 2026

  Large reasoning models (LRMs) aim to solve diverse and complex problems through structured reasoning. Recent advances in group-based policy optimization methods have shown promise in enabling stable advantage estimation without reliance on process-level annotations. However, these methods rely on ad…