Bayesian Methods and Mixture Models

Latest papers

• Fractionally Supervised Classification with Maxima Nominated Samples

  Mohammad Jafari Jozani, Jingyu Wang · 29 April 2026

  Fractionally supervised classification (FSC) offers a flexible framework for combining labeled and unlabeled data in model-based classification, but existing formulations assume simple random sampling. In many applications, however, the retained observation is an extreme order statistic from a set r…

• Mixed Membership sub-Gaussian Models

  Huan Qing · 27 April 2026

  The Gaussian mixture model is widely used in unsupervised learning, owing to its simplicity and interpretability. However, a fundamental limitation of the classical Gaussian mixture model is that it forces each observation to belong to exactly one component. In many practical applications, such as g…

• Fast estimation of Gaussian mixture components via centering and singular value thresholding

  Huan Qing · 22 April 2026

  Estimating the number of components is a fundamental challenge in unsupervised learning, particularly when dealing with high-dimensional data with many components or severely imbalanced component sizes. This paper addresses this challenge for classical Gaussian mixture models. The proposed estimator…

• How to Approximate Inference with Subtractive Mixture Models

  Lena Zellinger, Nicola Branchini, Lennert De Smet, V\'ictor Elvira, Nikolay Malkin, Antonio Vergari · 21 April 2026

  Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance sampling (IS). Recently, mixture models with negative coefficients, called subtractive mixture models (SMMs), have been proposed as a potentially mo…

• Active Statistical Inference

  Tijana Zrnic, Emmanuel J. Cand\`es · 9 April 2026

  Inspired by the concept of active learning, we propose active inference$\unicode{x2013}$a methodology for statistical inference with machine-learning-assisted data collection. Assuming a budget on the number of labels that can be collected, the methodology uses a machine learning model to identify w…

• Individual-heterogeneous sub-Gaussian Mixture Models

  Huan Qing · 8 April 2026

  The classical Gaussian mixture model assumes homogeneity within clusters, an assumption that often fails in real-world data where observations naturally exhibit varying scales or intensities. To address this, we introduce the individual-heterogeneous sub-Gaussian mixture model, a flexible framework …

• Regional climate risk assessment from climate models using probabilistic machine learning

  Zhong Yi Wan, Ignacio Lopez-Gomez, Robert Carver, Tapio Schneider, John Anderson, Fei Sha, Leonardo Zepeda-N\'u\~nez · 8 April 2026

  Effective climate risk assessment is hindered by the resolution gap between coarse global climate models and the fine-scale information needed for regional decisions. We introduce GenFocal, an AI framework that generates statistically accurate, fine-scale weather from coarse climate projections, wit…

• Multirate Stein Variational Gradient Descent for Efficient Bayesian Sampling

  Arash Sarshar · 7 April 2026

  Many particle-based Bayesian inference methods use a single global step size for all parts of the update. In Stein variational gradient descent (SVGD), however, each update combines two qualitatively different effects: attraction toward high-posterior regions and repulsion that preserves particle di…

• Symbolic Density Estimation: A Decompositional Approach

  Angelo Rajendram, Xieting Chu, Vijay Ganesh, Max Fieg, Aishik Ghosh · 31 March 2026

  We introduce AI-Kolmogorov, a novel framework for Symbolic Density Estimation (SymDE). Symbolic regression (SR) has been effectively used to produce interpretable models in standard regression settings but its applicability to density estimation tasks has largely been unexplored. To address the SymD…

• Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models

  Anupreet Porwal, Abel Rodriguez · 24 March 2026

  This paper introduces Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models. These priors are extensions of traditional mixtures of $g$ priors that allow for differential shrinkage for various (data-selected) blocks of parameters while fully accounting fo…

• On Consistency of Signature Using Lasso

  Xin Guo, Binnan Wang, Ruixun Zhang, Chaoyi Zhao · 24 March 2026

  Signatures are iterated path integrals of continuous and discrete-time processes, and their universal nonlinearity linearizes the problem of feature selection in time series data analysis. This paper studies the consistency of signature using Lasso regression, both theoretically and numerically. We …

• Model Selection and Parameter Estimation of Multi-dimensional Gaussian Mixture Model

  Xinyu Liu, Hai Zhang · 23 March 2026

  In this paper, we study the problem of learning multi-dimensional Gaussian Mixture Models (GMMs), with a specific focus on model order selection and efficient mixing distribution estimation. We first establish an information-theoretic lower bound on the critical sample complexity required for reliab…

• Amortized Bayesian Mixture Models

  \v{S}imon Kucharsk\'y, Paul Christian B\"urkner · 17 March 2026

  Finite mixtures are a broad class of models useful in scenarios where observed data is generated by multiple distinct processes but without explicit information about the responsible process for each data point. Estimating Bayesian mixture models is computationally challenging due to issues such as …

• Characterizing Evolution in Expectation-Maximization Estimates for Overspecified Mixed Linear Regression

  Zhankun Luo, Abolfazl Hashemi · 9 March 2026

  Mixture models have attracted significant attention due to practical effectiveness and comprehensive theoretical foundations. A persisting challenge is model misspecification, which occurs when the model to be fitted has more mixture components than those in the data distribution. In this paper, we …

• Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling

  Matthieu Blanke, Yongquan Qu, Sara Shamekh, Pierre Gentine · 4 March 2026

  Deep generative models hold great promise for representing complex physical systems, but their deployment is currently limited by the lack of guarantees on the physical plausibility of the generated outputs. Ensuring that known physical constraints are enforced is therefore critical when applying ge…

• Fourier Analysis on the Boolean Hypercube via Hoeffding Functional Decomposition

  Baptiste Ferrere (EDF R\&D PRISME, IMT, SINCLAIR AI Lab), Nicolas Bousquet (EDF R\&D PRISME, SINCLAIR AI Lab, LPSM), Fabrice Gamboa (IMT, ANITI), Jean-Michel Loubes (IMT, ANITI, REGALIA), Joseph Mur\'e (EDF R\&D PRISME) · 3 March 2026

  Fourier analysis on the Boolean hypercube is fundamentally defined as the orthogonal decomposition of the space of pseudo-Boolean functions with respect to the uniform probability measure. In this work, we propose an ANOVA-based generalization of the Fourier decomposition on the Boolean hypercube en…

• VICatMix: variational Bayesian clustering and variable selection for discrete biomedical data

  Jackie Rao, Paul D. W. Kirk · 3 March 2026

  Effective clustering of biomedical data is crucial in precision medicine, enabling accurate stratifiction of patients or samples. However, the growth in availability of high-dimensional categorical data, including `omics data, necessitates computationally efficient clustering algorithms. We present …

• Learning to Play Multi-Follower Bayesian Stackelberg Games

  Gerson Personnat, Tao Lin, Safwan Hossain, David C. Parkes · 3 March 2026

  In a multi-follower Bayesian Stackelberg game, a leader plays a mixed strategy over $L$ actions to which $n\ge 1$ followers, each having one of $K$ possible private types, best respond. The leader's optimal strategy depends on the distribution of the followers' private types. We study an online lear…

• In-Context Learning of Temporal Point Processes with Foundation Inference Models

  David Berghaus, Patrick Seifner, Kostadin Cvejoski, C\'esar Ojeda, Rams\'es J. S\'anchez · 2 March 2026

  Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamical rules and predict future events. Current neural network approaches to MTPP inference rely on training separate, specialized models for each target sy…

• Fair Model-based Clustering

  Jinwon Park, Kunwoong Kim, Jihu Lee, Yongdai Kim · 26 February 2026

  The goal of fair clustering is to find clusters such that the proportion of sensitive attributes (e.g., gender, race, etc.) in each cluster is similar to that of the entire dataset. Various fair clustering algorithms have been proposed that modify standard K-means clustering to satisfy a given fairn…

• Model Selection and Parameter Estimation of One-Dimensional Gaussian Mixture Models

  Xinyu Liu, Hai Zhang · 24 February 2026

  In this paper, we study the problem of learning one-dimensional Gaussian mixture models (GMMs) with a specific focus on estimating both the model order and the mixing distribution from independent and identically distributed (i.i.d.) samples. This paper establishes the optimal sampling complexity fo…

• Vectorized Bayesian Inference for Latent Dirichlet-Tree Allocation

  Zheng Wang, Nizar Bouguila · 24 February 2026

  Latent Dirichlet Allocation (LDA) is a foundational model for discovering latent thematic structure in discrete data, but its Dirichlet prior cannot represent the rich correlations and hierarchical relationships often present among topics. We introduce the framework of Latent Dirichlet-Tree Allocati…

• A spectral mixture representation of isotropic kernels with application to random Fourier features

  Nicolas Langren\'e, Xavier Warin, Pierre Gruet · 24 February 2026

  Rahimi and Recht (2007) introduced the idea of decomposing positive definite shift-invariant kernels by randomly sampling from their spectral distribution for machine learning applications. This famous technique, known as Random Fourier Features (RFF), is in principle applicable to any such kernel w…

• Analytical Results for Two Exponential Family Distributions in Hierarchical Dirichlet Processes

  Naiqi Li · 16 February 2026

  The Hierarchical Dirichlet Process (HDP) provides a flexible Bayesian nonparametric framework for modeling grouped data with a shared yet unbounded collection of mixture components. While existing applications of the HDP predominantly focus on the Dirichlet-multinomial conjugate structure, the frame…

• Variational phylogenetic inference with products over bipartitions

  Evan Sidrow, Alexandre Bouchard-C\^ot\'e, Lloyd T. Elliott · 16 February 2026

  Bayesian phylogenetics is vital for understanding evolutionary dynamics, and requires accurate and efficient approximation of posterior distributions over trees. In this work, we develop a variational Bayesian approach for ultrametric phylogenetic trees. We present a novel variational family based o…