Statistical and Computational Modeling

Derniers papiers

• Revisiting scaling laws for reward optimization

  Ali Aouad, Aymane El Gadarri, Vivek F. Farias · 1 octobre 2026

  Scaling laws for optimization against reward models in AI alignment have pinned down how performance depends on optimization effort---measured by a KL-divergence budget relative to a reference policy. Beyond a certain budget, over-optimization (or reward hacking) can arise: because we optimize again…

• OpenJev-RLCD: A Working RLCD Implementation

  Zhimin Gao, Pichao Wang · 1 octobre 2026

  Decision models such as Jev answer questions with probabilities, which are only useful if they are calibrated. Open-source reproductions rely on supervised fine-tuning plus temperature scaling, while reinforcement learning from verifiable rewards (RLVR) makes reasoning models overconfident. We prese…

• Diffusion Reward Models

  Xiangyang Wang, Bingxiang He, Zeyuan Liu, Jiaze WangZiqing Qiao, Yuxin Zuo, Huan-ang Gao, Cheng Qian, Wenbin Zhang, Ran Li, Youbang Sun, Ning Ding, Yuanchun Shi, Zhiyuan Liu, Chaojun Xiao, Chun Yu · 29 septembre 2026

  Reward models underpin the alignment of large language models, yet the dominant designs reduce each prompt--response pair to a point estimate or to a distribution from a fixed parametric family. This is at odds with human preference, which is inherently multimodal: the same response can be reasonabl…

• ExpBoN: Exponential-Noise Best-of-$n$ for Efficient Test-Time LLM Alignment

  Yanxiao Liu, Sicheng Wan, Deniz G\"und\"uz · 21 septembre 2026

  Best-of-$n$ (BoN) sampling is a simple yet effective inference-time alignment method, but hard maximization provides only coarse control over the trade-off between reward and distribution shift. Soft Best-of-$n$ (Verdun et al. 2025) provides smoother control and converges to the optimal distribution…

• Advantage Scale Calibration Imbalance in Group-Relative Optimization under Low-Variance Rewards: Diagnosis and Bounded Recovery

  Fei Ding · 18 septembre 2026

  In verifier-style RLVR, group-relative optimization often treats advantage scale as an implementation detail. This paper separates two low-variance cases: sub-resolution jitter that should not become a preference signal, and credible but small cardinal gaps that should be learned without distorting …

• Adaptive Margin Ordinal Loss: Penalizing Center-Class Hedging in Ordinal Classification

  Manisha Kandel · 11 septembre 2026

  Standard cross-entropy loss causes neural networks trained on ordinal classification tasks to hedge predictions toward center classes, a failure mode we term \emph{center-class hedging}. This occurs because predicting the middle class minimizes expected symmetric loss, making it the path of least re…

• Let Confidence Change, Not the Prediction: Prediction-Preserving Repair for Post-hoc Calibration

  Daehwan Kim, Haejun Chung, Ikbeom Jang · 2 septembre 2026

  Post-hoc calibration corrects reported confidence, yet a multiclass calibrator can also change the associated top-1 prediction. Accuracy captures only the net effect of these changes on correctness, not how often predictions change; the Top-1 Prediction Change Rate (TPCR) instead measures this frequ…

• How Proper Scoring Rules Shape LLM Forecasting

  Benjamin Turtel, Paul Wilczewski, Kris Skotheim, Ville A. Satop\"a\"a, Philip E. Tetlock · 31 août 2026

  This paper evaluates how reward function choice shapes the performance and behavior of LLM forecasters. We compare five proper scoring rules as training objectives for binary forecasts of resolved real-world events. Although the rules share the same theoretical incentive for truthful probability rep…

• Ordinal-Aware Calibration for Ordinal Classification

  Daehwan Kim, Haejun Chung, Ikbeom Jang · 17 août 2026

  Deep neural networks frequently produce overconfident, miscalibrated predictions. In ordinal classification, predictions must also adhere to a unimodal and order-consistent structure, a requirement that has dominated prior work while overlooking calibration. We formalize this joint challenge as ordi…

• Physiome-ODE: A Benchmark for Irregularly Sampled Multivariate Time Series Forecasting Based on Biological ODEs

  Christian Kl\"otergens, Vijaya Krishna Yalavarthi, Randolf Scholz, Maximilian Stubbemann, Stefan Born, Lars Schmidt-Thieme · 25 mai 2026

  State-of-the-art methods for forecasting irregularly sampled time series with missing values predominantly rely on just four datasets and a few small toy examples for evaluation. While ordinary differential equations (ODE) are the prevalent models in science and engineering, a baseline model that fo…

• LLM-Based Agents for Competitive Landscape Mapping in Drug Asset Due Diligence

  Vlad Vinogradov (Optic Inc), Alisa Vinogradova (AI Expert), Dmitrii Radkevich (Optic Inc), Ilya Yasny (Optic Inc), Dmitry Kobyzev (Optic Inc), Ivan Izmailov (Optic Inc), Katsiaryna Yanchanka (Optic Inc), Roman Doronin (Optic Inc), Andrey Doronichev (Optic Inc) · 11 mai 2026

  In this paper, we describe and benchmark a competitor-discovery component used within an agentic AI system for fast drug asset due diligence. A competitor-discovery AI agent, given an indication, retrieves all drugs comprising the competitive landscape of that indication and extracts canonical attri…

• LLM Probability Concentration: How Alignment Shrinks the Generative Horizon

  Chenghao Yang, Sida Li, Ari Holtzman · 4 mars 2026

  Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investigate this phenomenon through the lens of probability concentration in the model's output distribution. To quantify this co…

• Mechanistic Interpretability as Statistical Estimation: A Variance Analysis

  Maxime M\'eloux, Fran\c{c}ois Portet, Maxime Peyrard · 4 février 2026

  Mechanistic Interpretability (MI) aims to reverse-engineer model behaviors by identifying functional sub-networks. Yet, the scientific validity of these findings depends on their stability. In this work, we argue that circuit discovery is not a standalone task but a statistical estimation problem bu…

• GRAM: A Generative Foundation Reward Model for Reward Generalization

  Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, Murun Yang, Bei Li, Tong Xiao, Chunliang Zhang, Tongran Liu, Jingbo Zhu · 27 janvier 2026

  In aligning large language models (LLMs), reward models have played an important role, but are standardly trained as discriminative models and rely only on labeled human preference data. In this paper, we explore methods that train reward models using both unlabeled and labeled data. Building on the…

• Neural Logic Networks for Interpretable Classification

  Vincent Perreault, Katsumi Inoue, Richard Labib, Alain Hertz · 26 janvier 2026

  Traditional neural networks have an impressive classification performance, but what they learn cannot be inspected, verified or extracted. Neural Logic Networks on the other hand have an interpretable structure that enables them to learn a logical mechanism relating the inputs and outputs with AND a…

• Controlling Ensemble Variance in Diffusion Models: An Application for Reanalyses Downscaling

  Fabio Merizzi, Davide Evangelista, Harilaos Loukos · 15 janvier 2026

  In recent years, diffusion models have emerged as powerful tools for generating ensemble members in meteorology. In this work, we demonstrate how a Denoising Diffusion Implicit Model (DDIM) can effectively control ensemble variance by varying the number of diffusion steps. Introducing a theoretical …

• Deferred Poisoning: Making the Model More Vulnerable via Hessian Singularization

  Yuhao He, Jinyu Tian, Xianwei Zheng, Li Dong, Yuanman Li, Jiantao Zhou · 12 décembre 2025

  Recent studies have shown that deep learning models are very vulnerable to poisoning attacks. Many defense methods have been proposed to address this issue. However, traditional poisoning attacks are not as threatening as commonly believed. This is because they often cause differences in how the mod…

• Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learning

  Zijun Chen, Shengbo Wang, Nian Si · 4 novembre 2025

  Motivated by practical applications where stable long-term performance is critical-such as robotics, operations research, and healthcare-we study the problem of distributionally robust (DR) average-reward reinforcement learning. We propose two algorithms that achieve near-optimal sample complexity. …

• Debiasing Reward Models by Representation Learning with Guarantees

  Ignavier Ng, Patrick Bl\"obaum, Siddharth Bhandari, Kun Zhang, Shiva Kasiviswanathan · 29 octobre 2025

  Recent alignment techniques, such as reinforcement learning from human feedback, have been widely adopted to align large language models with human preferences by learning and leveraging reward models. In practice, these models often exploit spurious correlations, involving, e.g., response length, d…

• Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences

  Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao · 28 octobre 2025

  Reward models (RMs) play a critical role in aligning AI behaviors with human preferences, yet they face two fundamental challenges: (1) Modality Imbalance, where most RMs are mainly focused on text and image modalities, offering limited support for video, audio, and other modalities; and (2) Prefere…