Reinforcement Learning in Robotics

Últimos artículos

• Quantile Reward Policy Optimization: Alignment with Pointwise Regression and Exact Partition Functions

  Simon Matrenok, Skander Moalla, Caglar Gulcehre · 2 de diciembre de 2025

  Aligning large language models with pointwise absolute rewards has so far required online, on-policy algorithms such as PPO and GRPO. In contrast, simpler methods that can leverage offline or off-policy data, such as DPO and REBEL, are limited to learning from preference pairs or relative signals. T…

• State Entropy Regularization for Robust Reinforcement Learning

  Yonatan Ashlag, Uri Koren, Mirco Mutti, Esther Derman, Pierre-Luc Bacon, Shie Mannor · 2 de diciembre de 2025

  State entropy regularization has empirically shown better exploration and sample complexity in reinforcement learning (RL). However, its theoretical guarantees have not been studied. In this paper, we show that state entropy regularization improves robustness to structured and spatially correlated p…

• Much Ado About Noising: Dispelling the Myths of Generative Robotic Control

  Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang, Claire Jin, Daniel Pfrommer, Chenyang Yuan, Frank Permenter, Guannan Qu, Nicholas Boffi, Guanya Shi, Max Simchowitz · 2 de diciembre de 2025

  Generative models, like flows and diffusions, have recently emerged as popular and efficacious policy parameterizations in robotics. There has been much speculation as to the factors underlying their successes, ranging from capturing multi-modal action distribution to expressing more complex behavio…

• The Silence that Speaks: Neural Estimation via Communication Gaps

  Shubham Aggarwal, Dipankar Maity, Tamer Ba\c{s}ar · 2 de diciembre de 2025

  Accurate remote state estimation is a fundamental component of many autonomous and networked dynamical systems, where multiple decision-making agents interact and communicate over shared, bandwidth-constrained channels. These communication constraints introduce an additional layer of complexity, nam…

• Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning

  Qi Wang, Mian Wu, Yuyang Zhang, Mingqi Yuan, Wenyao Zhang, Haoxiang You, Yunbo Wang, Xin Jin, Xiaokang Yang, Wenjun Zeng · 2 de diciembre de 2025

  Reinforcement Learning (RL) has achieved remarkable success in various domains, yet it often relies on carefully designed programmatic reward functions to guide agent behavior. Designing such reward functions can be challenging and may not generalize well across different tasks. To address this limi…

• Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments

  Junwoo Chang, Minwoo Park, Joohwan Seo, Roberto Horowitz, Jongmin Lee, Jongeun Choi · 2 de diciembre de 2025

  Group symmetries provide a powerful inductive bias for reinforcement learning (RL), enabling efficient generalization across symmetric states and actions via group-invariant Markov Decision Processes (MDPs). However, real-world environments almost never realize fully group-invariant MDPs; dynamics, …

• Upcycled and Merged MoE Reward Model for Mitigating Reward Hacking

  Lingling Fu · 2 de diciembre de 2025

  Reward models play a critical role in Reinforcement Learning from Human Feedback (RLHF) by assessing the consistency between generated outputs and human preferences. However, conventional reward models are prone to reward hacking or over-optimization, where the policy exploits shortcut patterns to o…

• Learning Causal States Under Partial Observability and Perturbation

  Na Li, Hangguan Shan, Wei Ni, Wenjie Zhang, Xinyu Li, Yamin Wang · 2 de diciembre de 2025

  A critical challenge for reinforcement learning (RL) is making decisions based on incomplete and noisy observations, especially in perturbed and partially observable Markov decision processes (P$^2$OMDPs). Existing methods fail to mitigate perturbations while addressing partial observability. We pro…

• Sample-Efficient Tabular Self-Play for Offline Robust Reinforcement Learning

  Na Li, Zewu Zheng, Wei Ni, Hangguan Shan, Wenjie Zhang, Xinyu Li · 2 de diciembre de 2025

  Multi-agent reinforcement learning (MARL), as a thriving field, explores how multiple agents independently make decisions in a shared dynamic environment. Due to environmental uncertainties, policies in MARL must remain robust to tackle the sim-to-real gap. We focus on robust two-player zero-sum Mar…

• Provable Memory Efficient Self-Play Algorithm for Model-free Reinforcement Learning

  Na Li, Yuchen Jiao, Hangguan Shan, Shefeng Yan · 2 de diciembre de 2025

  The thriving field of multi-agent reinforcement learning (MARL) studies how a group of interacting agents make decisions autonomously in a shared dynamic environment. Existing theoretical studies in this area suffer from at least two of the following obstacles: memory inefficiency, the heavy depende…

• Symmetric Behavior Regularized Policy Optimization

  Lingwei Zhu, Haseeb Shah, Zheng Chen, Yukie Nagai, Martha White · 2 de diciembre de 2025

  Behavior Regularized Policy Optimization (BRPO) leverages asymmetric (divergence) regularization to mitigate the distribution shift in offline Reinforcement Learning. This paper is the first to study the open question of symmetric regularization. We show that symmetric regularization does not permit…

• Outcome-based Reinforcement Learning to Predict the Future

  Benjamin Turtel, Danny Franklin, Kris Skotheim, Luke Hewitt, Philipp Schoenegger · 2 de diciembre de 2025

  Reinforcement Learning with Verifiable Rewards (RLVR) has been an effective approach for improving Large Language Models' reasoning in domains such as coding and mathematics. Here, we apply RLVR methods towards forecasting future real-world events - a challenging task for RL due to the very noisy (a…

• Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments

  Riley Simmons-Edler, Ryan P. Badman, Felix Baastad Berg, Raymond Chua, John J. Vastola, Joshua Lunger, William Qian, Kanaka Rajan · 2 de diciembre de 2025

  Understanding the behavior of deep reinforcement learning (DRL) agents -particularly as task and agent sophistication increase- requires more than simple comparison of reward curves, yet standard methods for behavioral analysis remain underdeveloped in DRL. We apply tools from neuroscience and ethol…

• A Diffusion Model Framework for Maximum Entropy Reinforcement Learning

  Sebastian Sanokowski, Kaustubh Patil, Alois Knoll · 2 de diciembre de 2025

  Diffusion models have achieved remarkable success in data-driven learning and in sampling from complex, unnormalized target distributions. Building on this progress, we reinterpret Maximum Entropy Reinforcement Learning (MaxEntRL) as a diffusion model-based sampling problem. We tackle this problem b…

• RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies

  Guillermo Garcia-Cobo, Maximilian Igl, Peter Karkus, Zhejun Zhang, Michael Watson, Yuxiao Chen, Boris Ivanovic, Marco Pavone · 2 de diciembre de 2025

  Autonomous driving policies are typically trained via open-loop behavior cloning of human demonstrations. However, such policies suffer from covariate shift when deployed in closed loop, leading to compounding errors. We introduce Rollouts as Demonstrations (RoaD), a simple and efficient method to m…

• Forecasting in Offline Reinforcement Learning for Non-stationary Environments

  Suzan Ece Ada, Georg Martius, Emre Ugur, Erhan Oztop · 2 de diciembre de 2025

  Offline Reinforcement Learning (RL) provides a promising avenue for training policies from pre-collected datasets when gathering additional interaction data is infeasible. However, existing offline RL methods often assume stationarity or only consider synthetic perturbations at test time, assumption…

• InnoGym: Benchmarking the Innovation Potential of AI Agents

  Jintian Zhang, Kewei Xu, Jingsheng Zheng, Zhuoyun Yu, Yuqi Zhu, Yujie Luo, Lanning Wei, Shuofei Qiao, Lun Du, Da Zheng, Shumin Deng, Huajun Chen, Ningyu Zhang · 2 de diciembre de 2025

  LLMs and Agents have achieved impressive progress in code generation, mathematical reasoning, and scientific discovery. However, existing benchmarks primarily measure correctness, overlooking the diversity of methods behind solutions. True innovation depends not only on producing correct answers but…

• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices

  Chujie Zheng, Kai Dang, Bowen Yu, Mingze Li, Huiqiang Jiang, Junrong Lin, Yuqiong Liu, An Yang, Jingren Zhou, Junyang Lin · 2 de diciembre de 2025

  This paper proposes a novel formulation for reinforcement learning (RL) with large language models, explaining why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE. Specifically, through a fir…

• World Model Robustness via Surprise Recognition

  Geigh Zollicoffer, Tanush Chopra, Mingkuan Yan, Xiaoxu Ma, Kenneth Eaton, Mark Riedl · 2 de diciembre de 2025

  AI systems deployed in the real world must contend with distractions and out-of-distribution (OOD) noise that can destabilize their policies and lead to unsafe behavior. While robust training can reduce sensitivity to some forms of noise, it is infeasible to anticipate all possible OOD conditions. T…

• AltNet: Addressing the Plasticity-Stability Dilemma in Reinforcement Learning

  Mansi Maheshwari, John C. Raisbeck, Bruno Castro da Silva · 2 de diciembre de 2025

  Neural networks have shown remarkable success in supervised learning when trained on a single task using a fixed dataset. However, when neural networks are trained on a reinforcement learning task, their ability to continue learning from new experiences declines over time. This decline in learning a…

• List Replicable Reinforcement Learning

  Bohan Zhang, Michael Chen, A. Pavan, N. V. Vinodchandran, Lin F. Yang, Ruosong Wang · 2 de diciembre de 2025

  Replicability is a fundamental challenge in reinforcement learning (RL), as RL algorithms are empirically observed to be unstable and sensitive to variations in training conditions. To formally address this issue, we study \emph{list replicability} in the Probably Approximately Correct (PAC) RL fram…

• Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction

  Arad Firouzkouhi (University of Southern California), Omid Mirzaeedodangeh (ETH Z\"urich), Lars Lindemann (ETH Z\"urich) · 2 de diciembre de 2025

  Active imitation learning (AIL) combats covariate shift by querying an expert during training. However, expert action labeling often dominates the cost, especially in GPU-intensive simulators, human-in-the-loop settings, and robot fleets that revisit near-duplicate states. We present Conformalized R…

• Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworks

  Pei Zhou, Wanting Yao, Qian Luo, Xunzhe Zhou, Yanchao Yang · 2 de diciembre de 2025

  Goal-conditioned policy learning for robotic manipulation presents significant challenges in maintaining performance across diverse objectives and environments. We introduce Hyper-GoalNet, a framework that generates task-specific policy network parameters from goal specifications using hypernetworks…

• SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning

  Taewook Nam, Sung Ju Hwang · 2 de diciembre de 2025

  Recent advances in robotic policy learning have enabled complex manipulation in real-world environments, yet the execution speed of these policies often lags behind hardware capabilities due to the cost of collecting faster demonstrations. Existing works on policy acceleration reinterpret action seq…

• Arbitrary Entropy Policy Optimization Breaks The Exploration Bottleneck of Reinforcement Learning

  Chen Wang, Zhaochun Li, Jionghao Bai, Yuzhi Zhang, Shisheng Cui, Zhou Zhao, Yue Wang · 2 de diciembre de 2025

  Reinforcement Learning (RL) is essential for enhancing the reasoning capabilities of large language models (LLMs), yet the widely adopted Group Relative Policy Optimization (GRPO) suffers from entropy collapse, causing exploration to vanish and policies to converge prematurely. As a result, RL is wi…