Reinforcement Learning in Robotics

Últimos artículos

• Calibration-Aware Policy Optimization for Reasoning LLMs

  Ziqi Wang, Xingzhou Lou, Meiqi Wu, Zhengqi Wen, Junge Zhang · 15 de abril de 2026

  Group Relative Policy Optimization (GRPO) enhances LLM reasoning but often induces overconfidence, where incorrect responses yield lower perplexity than correct ones, degrading relative calibration as described by the Area Under the Curve (AUC). Existing approaches either yield limited improvements …

• Social Learning Strategies for Evolved Virtual Soft Robots

  K. Ege de Bruin, Kyrre Glette, Kai Olav Ellefsen, Giorgia Nadizar, Eric Medvet · 15 de abril de 2026

  Optimizing the body and brain of a robot is a coupled challenge: the morphology determines what control strategies are effective, while the control parameters influence how well the morphology performs. This joint optimization can be done through nested loops of evolutionary and learning processes, …

• Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring

  Melvin Laux, Yi-Ling Liu, Rina Alo, S\"oren T\"opper, Mariela De Lucas Alvarez, Frank Kirchner, Rebecca Adam · 15 de abril de 2026

  Although autonomous underwater vehicles promise the capability of marine ecosystem monitoring, their deployment is fundamentally limited by the difficulty of controlling vehicles under highly uncertain and non-stationary underwater dynamics. To address these challenges, we employ a data-driven reinf…

• Free Random Projection for In-Context Reinforcement Learning

  Tomohiro Hayase, Beno\^it Collins, Nakamasa Inoue · 15 de abril de 2026

  Hierarchical inductive biases are hypothesized to promote generalizable policies in reinforcement learning, as demonstrated by explicit hyperbolic latent representations and architectures. Therefore, a more flexible approach is to have these biases emerge naturally from the algorithm. We introduce F…

• Self-Monitoring Benefits from Structural Integration: Lessons from Metacognition in Continuous-Time Multi-Timescale Agents

  Ying Xie · 15 de abril de 2026

  Self-monitoring capabilities -- metacognition, self-prediction, and subjective duration -- are often proposed as useful additions to reinforcement learning agents. But do they actually help? We investigate this question in a continuous-time multi-timescale agent operating in predator-prey survival e…

• Replicable Reinforcement Learning with Linear Function Approximation

  Eric Eaton, Marcel Hussing, Michael Kearns, Aaron Roth, Sikata Bela Sengupta, Jessica Sorrell · 15 de abril de 2026

  Replication of experimental results has been a challenge faced by many scientific disciplines, including the field of machine learning. Recent work on the theory of machine learning has formalized replicability as the demand that an algorithm produce identical outcomes when executed twice on differe…

• Safety Training Modulates Harmful Misalignment Under On-Policy RL, But Direction Depends on Environment Design

  Leon Eshuijs, Shihan Wang, Antske Fokkens · 15 de abril de 2026

  Specification gaming under Reinforcement Learning (RL) is known to cause LLMs to develop sycophantic, manipulative, or deceptive behavior, yet the conditions under which this occurs remain unclear. We train 11 instruction-tuned LLMs (0.5B--14B) with on-policy RL across 3 environments and find that m…

• Robust Optimization for Mitigating Reward Hacking with Correlated Proxies

  Zixuan Liu, Xiaolin Sun, Zizhan Zheng · 15 de abril de 2026

  Designing robust reinforcement learning (RL) agents in the presence of imperfect reward signals remains a core challenge. In practice, agents are often trained with proxy rewards that only approximate the true objective, leaving them vulnerable to reward hacking, where high proxy returns arise from …

• Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents

  Hao Wang, Guozhi Wang, Han Xiao, Yufeng Zhou, Yue Pan, Jichao Wang, Ke Xu, Yafei Wen, Xiaohu Ruan, Xiaoxin Chen, Honggang Qi · 14 de abril de 2026

  Reinforcement learning (RL) has been widely used to train LLM agents for multi-turn interactive tasks, but its sample efficiency is severely limited by sparse rewards and long horizons. On-policy self-distillation (OPSD) alleviates this by providing dense token-level supervision from a privileged te…

• Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning

  Siyuan Xu, Shiyang Li, Xin Liu, Tianyi Liu, Yixiao Li, Zhan Shi, Zixuan Zhang, Zilong Wang, Qingyu Yin, Jianshu Chen, Tuo Zhao, Bing Yin · 14 de abril de 2026

  Existing synthetic tool-use corpora are primarily designed for offline supervised fine-tuning, yet reinforcement learning (RL) requires executable environments that support reward-checkable online rollouts. We propose COVERT, a two-stage pipeline that first generates reliable base tool-use trajector…

• B\'ezierFlow: Learning B\'ezier Stochastic Interpolant Schedulers for Few-Step Generation

  Yunhong Min, Juil Koo, Seungwoo Yoo, Minhyuk Sung · 14 de abril de 2026

  We introduce B\'ezierFlow, a lightweight training approach for few-step generation with pretrained diffusion and flow models. B\'ezierFlow achieves a 2-3x performance improvement for sampling with $\leq$ 10 NFEs while requiring only 15 minutes of training. Recent lightweight training approaches have…

• CROP: Conservative Reward for Model-based Offline Policy Optimization

  Hao Li, Xiao-Hu Zhou, Shu-Hai Li, Mei-Jiang Gui, Xiao-Liang Xie, Shi-Qi Liu, Shuang-Yi Wang, Zhen-Qiu Feng, Zeng-Guang Hou · 14 de abril de 2026

  Offline reinforcement learning (RL) aims to optimize a policy using collected data without online interactions. Model-based approaches are particularly appealing for addressing offline RL challenges because of their capability to mitigate the limitations of data coverage through data generation usin…

• TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distance

  Yuyang Liu, Chuan Wen, Yihang Hu, Dinesh Jayaraman, Yang Gao · 14 de abril de 2026

  Designing dense rewards is crucial for reinforcement learning (RL), yet in robotics it often demands extensive manual effort and lacks scalability. One promising solution is to view task progress as a dense reward signal, as it quantifies the degree to which actions advance the system toward task co…

• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

  Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield, Ankit Goyal, Stan Birchfield, Fabio Ramos, Jonathan Tremblay · 14 de abril de 2026

  The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a lack of true generalization testing. Existing benchmarks often exhibit significant domain overlap between training and ev…

• Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning

  Zikang Shan, Han Zhong, Liwei Wang, Li Zhao · 14 de abril de 2026

  Credit assignment is a central challenge in reinforcement learning (RL). Classical actor-critic methods address this challenge through fine-grained advantage estimation based on a learned value function. However, learned value models are often avoided in modern large language model (LLM) RL because …

• Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation

  Jiashu Yao, Heyan Huang, Zeming Liu, Yuhang Guo · 14 de abril de 2026

  To overcome the sparse reward challenge in reinforcement learning (RL) for agents based on large language models (LLMs), we propose Mutual Information Self-Evaluation (MISE), an RL paradigm that utilizes hindsight generative self-evaluation as dense reward signals while simultaneously calibrating th…

• A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning

  Ming Lei, Christophe Baehr · 14 de abril de 2026

  Reinforcement learning (RL) has become a key approach for enhancing reasoning in large language models (LLMs), yet scalable training is often hindered by the rapid collapse of policy entropy, which leads to premature convergence and performance saturation. This paper provides a comparative theoretic…

• Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers

  Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini · 14 de abril de 2026

  Prevalent reinforcement learning~(RL) methods for fine-tuning LLM reasoners, such as GRPO or Leave-one-out PPO, abandon the learned value function in favor of empirically estimated returns. This hinders test-time compute scaling that relies on using the value-function for verification. Yet if parall…

• Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning

  Yixuan Even Xu, Yash Savani, Fei Fang, J. Zico Kolter · 14 de abril de 2026

  Reinforcement learning with verifiable rewards (RLVR) has emerged as the leading approach for enhancing reasoning capabilities in large language models. However, it faces a fundamental compute and memory asymmetry: rollout generation is embarrassingly parallel and memory-light, whereas policy update…

• Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentation

  Xue Qin, Simin Luan, John See, Cong Yang, Zhijun Li · 14 de abril de 2026

  As embodied robots move toward fleet-scale operation, multi-robot coordination is becoming a central systems challenge. Existing approaches often treat this as motivation for increasing internal multi-agent decomposition within each robot. We argue for a different principle: multi-robot coordination…

• OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems

  Kun Liu, Liqun Chen · 14 de abril de 2026

  The alignment of Multi-Agent Systems (MAS) for autonomous software engineering is constrained by evaluator epistemic uncertainty. Current paradigms, such as Reinforcement Learning from Human Feedback (RLHF) and AI Feedback (RLAIF), frequently induce model sycophancy, while execution-based environmen…

• Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for Safe Reinforcement Learning in Optimal Control

  Roya Khalili Amirabadi, Mohsen Jalaeian Farimani, Omid Solaymani Fard · 14 de abril de 2026

  This paper proposes a novel reinforcement learning framework, named Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER), designed to achieve safe and scalable optimal control of nonlinear systems. The proposed SODACER mechanism consisting of a Fast-Buffer for rapid adaptation…

• Belief-State RWKV for Reinforcement Learning under Partial Observability

  Liu Xiao · 14 de abril de 2026

  We propose a stronger formulation of RL on top of RWKV-style recurrent sequence models, in which the fixed-size recurrent state is explicitly interpreted as a belief state rather than an opaque hidden vector. Instead of conditioning policy and value on a single summary h_t, we maintain a compact unc…

• Influencing Humans to Conform to Preference Models for RLHF

  Stephane Hatgis-Kessell, W. Bradley Knox, Serena Booth, Peter Stone · 14 de abril de 2026

  Designing a reinforcement learning from human feedback (RLHF) algorithm to approximate a human's unobservable reward function requires assuming, implicitly or explicitly, a model of human preferences. A preference model that poorly describes how humans generate preferences risks learning a poor appr…

• EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

  Xue Qin, Simin Luan, John See, Cong Yang, Zhijun Li · 14 de abril de 2026

  Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluation remains dominated by task success metrics such as completion rate or manipulation accuracy. These metrics leave a critical gap: they do not measure …