Gaussian Processes and Bayesian Inference

Latest papers

• Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods

  Saksham Kiroriwal, Julius Pfrommer, J\"urgen Beyerer · 28 September 2026

  We study Bayesian optimization (BO) through the lens of information geometry. Pulling back the Fisher information metric through the surrogate posterior map yields a local sensitivity tensor on the input space, which leads to an upper bound on the gradient of reparameterizable acquisition functions.…

• Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matrices

  Yanchuang Cao, Jun Liu, Tengchao Yu, Heng Yong · 28 September 2026

  Gaussian processes constitute a cornerstone of probabilistic machine learning, yet scaling them to large datasets typically forces a trade-off between computational efficiency and model fidelity. This work bridges this gap by presenting an adaptive multi-resolution Gaussian process framework that is…

• Residual Correlation as a Diagnostic for Joint-Uncertainty Gains from GP Coregionalisation

  Fangqin Zhou, Joaquin Vanschoren · 25 September 2026

  In multi-target regression, correlated targets are often coupled through multi-output Gaussian processes with an intrinsic model of coregionalisation (GP-ICM), assuming that sharing statistical strength improves overall performance. In practice, the benefits are inconsistent. Across the settings stu…

• Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

  Subhasish Basak, Anik Roy, Sourabh Bhattacharya · 24 September 2026

  We propose a Bayesian nonparametric mixture of regression trees with a Dirichlet process prior over tree-parameter pairs, enabling data-driven selection of ensemble size and unifying CART, BART, random forests, and boosting. A novel splitting rule driven by the posterior predictive of a Gaussian pro…

• Resource-Efficient Distributed Recursive Gaussian Processes

  Josephine King, Ali Emre Balci, Raj Thilak Rajan · 24 September 2026

  Gaussian processes (GPs) provide a flexible framework for learning unknown functions from noisy measurements while quantifying predictive uncertainty, making them well suited for estimation in multi-agent systems. However, when measurements are collected by multiple agents, maintaining a unified GP …

• On Basis Function Selection for Sparse Gaussian Process Regression

  Marnix Van Soom, Ivan De Boi · 23 September 2026

  Sparse Gaussian processes achieve $O(N)$ inference by replacing the kernel with an appropriate expansion in a fixed basis $\{\phi_j\}$ on the input space. Given a compute budget $M \ll N$, practitioners conventionally truncate the basis to its first $M$ entries. Nothing in the formalism, however, pr…

• Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

  Abhishek Srivastava, Arijit Hazra, Rajesh Dubbaku · 23 September 2026

  Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study …

• Triply-Scalable Equivariant Gaussian Process Modeling

  Tim Steinert, David Ginsbourger · 21 September 2026

  Gaussian processes (GPs) provide principled probabilistic predictions while encoding prior knowledge, including equivariances. Yet, their use in large-scale scientific problems is limited by computational cost. Equivariant neural networks are common but typically lack the uncertainty quantification …

• COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regression

  Zewen Yang, Xiaobing Dai, Zhenxiao Yin, Hang Zhao, Zhijun Li, C. C. Chan · 18 September 2026

  In this paper, we tackle the problem of jointly estimating the system states and partially unknown dynamics within distributed sensor-equipped networks, particularly in scenarios where only partial state observations are available. To address this issue, we propose an observer-based dynamic cooperat…

• Online Adaptive Kernel Mixing for Gaussian Process Decision Making

  Kavin Aravindan, Mani Tej Sriram, Gautam Dasarathy, Tejas Bodas · 18 September 2026

  Gaussian Processes (GPs) are widely used as surrogates for black-box functions in sequential decision-making problems such as Bayesian optimization (BO), level set estimation (LSE), and Bayesian active learning (BAL). GP performance critically depends on kernels, and standard kernels can lead to sub…

• Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

  Dier Tang (Department of Mathematics, The University of Hong Kong, Hong Kong, China), Guangyue Han (Department of Mathematics, The University of Hong Kong, Hong Kong, China) · 16 September 2026

  This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommodate noisy observations as well as attribute structures that may exhibit directional preferences. We introduce the notion of weak spherical symmetry, quantifie…

• Online Gradient Computation for Warping Gaussian Process Transformations

  Emilio Ruiz-Moreno, Konstantinos Slavakis, Baltasar Beferull-Lozano · 16 September 2026

  Warped Gaussian processes (GPs) handle non-Gaussian observations by mapping them into a latent standard GP via a parametric transformation called warping. Existing streaming variants, however, either optimize the warping parameters periodically or sacrifice analytical tractability for a higher model…

• Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization

  Luo Long, Coralia Cartis, Paz Fink Shustin · 14 September 2026

  Bayesian optimisation (BO) enables sample-efficient global optimisation of expensive black-box functions but remains challenging in high dimensions. We investigate nonlinear dimensionality reduction to a sequence of low-dimensional latent-space BO (LSBO) problems. Early LSBO used linear random and s…

• Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences

  Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, Bharath K. Sriperumbudur · 10 September 2026

  This monograph studies the relations between two approaches using positive definite kernels: probabilistic methods using Gaussian processes, and non-probabilistic methods using reproducing kernel Hilbert spaces (RKHS). They are widely studied and used in machine learning, statistics, and numerical a…

• Active Inference for an Intelligent Agent in Autonomous Reconnaissance Missions

  Johan Schubert, Farzad Kamrani, Tove Gustavi · 7 September 2026

  We develop an active inference route-planning method for the autonomous control of intelligent agents. The aim is to reconnoiter a geographical area to maintain a common operational picture. To achieve this, we construct an evidence map that reflects our current understanding of the situation, incor…

• No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

  Edvin Ketabati Augustinsson, Robert A. Bridges · 4 September 2026

  Gaussian-process Bayesian optimization (GP-BO) excels at black-box optimization of costly functions, e.g., hyperparameter optimization (HPO) and multi-agent system (MAS) design. Convergence-rate guarantees exist for select methods, notably GP upper confidence bound (GP-UCB), but require a fixed kern…

• Neural means and kernel corrections for operator learning

  Yitzchak Shmalo · 2 September 2026

  We combine neural network means with exact Mat\'ern kernel regressions of their residuals and of their learned features, and evaluate the pairing on two public emulation problems with published baselines: the structural-mechanics benchmark of de Hoop et al. and the OCO-2 radiative-transfer emulator …

• Simplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classification

  Bernardo Williams, Harsha Vardhan Tetali, Arto Klami, Marcelo Hartmann · 31 August 2026

  We propose a conjugate and calibrated Gaussian process (GP) model for multi-class classification by exploiting the geometry of the probability simplex. Our approach uses Aitchison geometry to map simplex-valued class probabilities to an unconstrained Euclidean representation, turning classification …

• Multi-output Gaussian process prediction of physical fields under linear equality constraints

  Mahamat Hamdan Nassouradine, Cl\'ement Gauchy, Pierre-Emmanuel Angeli, S\'ebastien da Veiga · 27 August 2026

  We address the simultaneous prediction of multiple high-dimensional physical fields governed by linear equality constraints, a setting that arises in many real-world applications in physics machine learning. Gaussian process (GP) regression is a widely used surrogate modeling approach due to its eff…

• Fast rates in Bayesian online learning with approximate posteriors

  Ilsang Ohn · 27 August 2026

  Exact Bayes prediction enjoys fast predictive regret guarantees, but exact posterior updating or representation may be too costly for online use. We study when these statistical guarantees are preserved by computational approximations. We show that the cumulative price of posterior approximation can…

• Simultaneous inference of environmental and interaction forces in collective dynamics

  Nipuni de Silva, Ming Zhong, James M. Greene · 27 August 2026

  Collective dynamics arise in a wide range of physical, biological, and engineering applications. Examples include cell migration, swarm robotics, social dynamics, and animal behavior. A defining characteristic of these systems is the emergence of large-scale coordination from local interactions amon…

• Variational Outlier-Robust Gaussian Process Regression with Generative Modeling

  Arslan Majal, Aamir Hussain Chughtai · 18 August 2026

  Outliers can substantially distort Gaussian process regression (GPR) due to its conventional Gaussian observation likelihood, leading to inaccurate model learning and prediction. To address this limitation, this article introduces a generative GPR model that captures observation-specific contaminati…

• Information Geometry of Message Passing

  Mykola Lukashchuk, Kyrylo Yemets, Alex Ledbetter, İsmail Şenöz · 18 August 2026

  We show that the natural-gradient stationary condition of variational inference has an edge-local form on a Forney-style factor graph. We start from the Bethe free energy and constrain a selected edge marginal to an exponential family. At a stationary point, the natural parameter of that edge equals…

• On the Brittleness of Maximum Likelihood Estimation for Gaussian Process Hyperparameter Optimization

  Tyler R. Johnson, Kian Ben-Jacob, Christopher P. Muller, Ramin Bostanabad · 17 August 2026

  Machine learning (ML) has become an indispensable part of modern engineering design workflows. A crucial step in training an ML model is the selection of the loss function which can be systematically formulated via various techniques such as maximum likelihood estimation (MLE) and cross-validation .…

• Finding the Needle in a Haystack: Test-Time Analog Circuit Representation Adaptation for Bayesian Optimization

  Fin Amin, Sounak Dutta, Paul D. Franzon · 14 August 2026

  Bayesian optimization (BO) is a sample-efficient framework for analog circuit topology search, where evaluating each candidate topology can require costly simulation. However, representation-based BO methods typically treat circuit embeddings as fixed after encoder training. This creates a mismatch …