Bayesian Methods and Mixture Models

Neueste Paper

• Improved sampling algorithms and functional inequalities for non-log-concave distributions

  Yuchen He, Zhehan Lei, Jianan Shao, Chihao Zhang · 10. Februar 2026

  We study the problem of sampling from a distribution $\mu$ with density $\propto e^{-V}$ for some potential function $V:\mathbb R^d\to \mathbb R$ with query access to $V$ and $\nabla V$. We start with the following standard assumptions: (1) $V$ is $L$-smooth. (2) The second moment $\mathbf{E}_{X…

• Understanding Generalization in Diffusion Distillation via Probability Flow Distance

  Huijie Zhang, Zijian Huang, Siyi Chen, Jinfan Zhou, Zekai Zhang, Peng Wang, Qing Qu · 10. Februar 2026

  Diffusion distillation provides an effective approach for learning lightweight and few-steps diffusion models with efficient generation. However, evaluating their generalization remains challenging: theoretical metrics are often impractical for high-dimensional data, while no practical metrics rigor…

• Transcendental Regularization of Finite Mixtures:Theoretical Guarantees and Practical Limitations

  Ernest Fokou\'e · 5. Februar 2026

  Finite mixture models are widely used for unsupervised learning, but maximum likelihood estimation via EM suffers from degeneracy as components collapse. We introduce transcendental regularization, a penalized likelihood framework with analytic barrier functions that prevent degeneracy while maintai…

• Accurate and scalable exchange-correlation with deep learning

  Giulia Luise, Chin-Wei Huang, Thijs Vogels, Derk P. Kooi, Sebastian Ehlert, Stephanie Lanius, Klaas J. H. Giesbertz, Amir Karton, Deniz Gunceler, Megan Stanley, Wessel P. Bruinsma, Lin Huang, Xinran Wei, Jos\'e Garrido Torres, Abylay Katbashev, Rodrigo Chavez Zavaleta, B\'alint M\'at\'e, S\'ekou-Oumar Kaba, Roberto Sordillo, Yingrong Chen, David B. Williams-Young, Christopher M. Bishop, Jan Hermann, Rianne van den Berg, Paola Gori-Giorgi · 5. Februar 2026

  Density Functional Theory (DFT) is the most widely used electronic structure method for predicting the properties of molecules and materials. Although DFT is, in principle, an exact reformulation of the Schr\"odinger equation, practical applications rely on approximations to the unknown exchange-cor…

• Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data

  Aayush Mishra, Daniel Habermann, Marvin Schmitt, Stefan T. Radev, Paul-Christian B\"urkner · 3. Februar 2026

  Amortized Bayesian inference (ABI) with neural networks can solve probabilistic inverse problems orders of magnitude faster than classical methods. However, ABI is not yet sufficiently robust for widespread and safe application. When performing inference on observations outside the scope of the simu…

• Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function

  Tuan Minh Pham, Thinh Cao, Viet Nguyen, Huy Nguyen, Nhat Ho, Alessandro Rinaldo · 3. Februar 2026

  The sigmoid gate in mixture-of-experts (MoE) models has been empirically shown to outperform the softmax gate across several tasks, ranging from approximating feed-forward networks to language modeling. Additionally, recent efforts have demonstrated that the sigmoid gate is provably more sample-effi…

• Emergence of Distortions in High-Dimensional Guided Diffusion Models

  Enrico Ventura, Beatrice Achilli, Luca Ambrogioni, Carlo Lucibello · 3. Februar 2026

  Classifier-free guidance (CFG) is the de facto standard for conditional sampling in diffusion models, yet it often leads to a loss of diversity in generated samples. We formalize this phenomenon as generative distortion, defined as the mismatch between the CFG-induced sampling distribution and the t…

• A VAE Approach to Sample Multivariate Extremes

  Nicolas Lafon, Philippe Naveau, Ronan Fablet · 2. Februar 2026

  Generating accurate extremes from an observational data set is crucial when seeking to estimate risks associated with the occurrence of future extremes which could be larger than those already observed. Applications range from the occurrence of natural disasters to financial crashes. Generative appr…

• Understanding Self-Supervised Learning via Gaussian Mixture Models

  Parikshit Bansal, Ali Kavis, Sujay Sanghavi · 30. Januar 2026

  Self-supervised learning attempts to learn representations from un-labeled data; it does so via a loss function that encourages the embedding of a point to be close to that of its augmentations. This simple idea performs remarkably well, yet it is not precisely theoretically understood why this is t…

• TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering

  Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang · 30. Januar 2026

  Clustering tabular data is a fundamental yet challenging problem due to heterogeneous feature types, diverse data-generating mechanisms, and the absence of transferable inductive biases across datasets. Prior-fitted networks (PFNs) have recently demonstrated strong generalization in supervised tabul…

• A Federated Generalized Expectation-Maximization Algorithm for Mixture Models with an Unknown Number of Components

  Michael Ibrahim, Nagi Gebraeel, Weijun Xie · 30. Januar 2026

  We study the problem of federated clustering when the total number of clusters $K$ across clients is unknown, and the clients have heterogeneous but potentially overlapping cluster sets in their local data. To that end, we develop FedGEM: a federated generalized expectation-maximization algorithm fo…

• Bias-Reduced Estimation of Finite Mixtures: An Application to Latent Group Structures in Panel Data

  Rapha\"el Langevin · 29. Januar 2026

  Finite mixture models are widely used in econometric analyses to capture unobserved heterogeneity. This paper shows that maximum likelihood estimation of finite mixtures of parametric densities can suffer from substantial finite-sample bias in all parameters under mild regularity conditions. The bia…

• Generative Modeling with Bayesian Sample Inference

  Marten Lienen, Marcel Kollovieh, Stephan G\"unnemann · 28. Januar 2026

  We derive a novel generative model from iterative Gaussian posterior inference. By treating the generated sample as an unknown variable, we can formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to iteratively na…

• Whitening Spherical Gaussian Mixtures in the Large-Dimensional Regime

  Mohammed Racim Moussa Boudjemaa, Alper Kalle, Xiaoyi Mai, Jos\'e Henrique de Morais Goulart, C\'edric F\'evotte · 22. Januar 2026

  Whitening is a classical technique in unsupervised learning that can facilitate estimation tasks by standardizing data. An important application is the estimation of latent variable models via the decomposition of tensors built from high-order moments. In particular, whitening orthogonalizes the mea…

• Semi-Supervised Mixture Models under the Concept of Missing at Radom with Margin Confidence and Aranda Ordaz Function

  Jinyang Liao, Ziyang Lyu · 22. Januar 2026

  This paper presents a semi-supervised learning framework for Gaussian mixture modelling under a Missing at Random (MAR) mechanism. The method explicitly parameterizes the missingness mechanism by modelling the probability of missingness as a function of classification uncertainty. To quantify classi…

• Factorizable joint shift revisited

  Dirk Tasche · 22. Januar 2026

  Factorizable joint shift (FJS) was proposed as a type of distribution shift (or dataset shift) that comprises both covariate and label shift. Recently, it has been observed that FJS actually arises from consecutive label and covariate (or vice versa) shifts. Research into FJS so far has been confine…

• Statistical Inference for Fuzzy Clustering

  Qiuyi Wu, Zihan Zhu, Anru R. Zhang · 7. Januar 2026

  Clustering is a central tool in biomedical research for discovering heterogeneous patient subpopulations, where group boundaries are often diffuse rather than sharply separated. Traditional methods produce hard partitions, whereas soft clustering methods such as fuzzy $c$-means (FCM) allow mixed mem…

• Fast Gibbs Sampling on Bayesian Hidden Markov Model with Missing Observations

  Dongrong Li, Tianwei Yu, Xiaodan Fan · 6. Januar 2026

  The Hidden Markov Model (HMM) is a widely-used statistical model for handling sequential data. However, the presence of missing observations in real-world datasets often complicates the application of the model. The EM algorithm and Gibbs samplers can be used to estimate the model, yet suffering fro…

• New affine invariant ensemble samplers and their dimensional scaling

  Yifan Chen · 1. Januar 2026

  We introduce new affine invariant ensemble Markov chain Monte Carlo (MCMC) samplers that are easy to construct and improve upon existing methods, especially for high-dimensional problems. We first propose a simple derivative-free side move sampler that improves upon popular samplers in the \texttt{e…

• Robust Unsupervised Multi-task and Transfer Learning on Gaussian Mixture Models

  Ye Tian, Haolei Weng, Lucy Xia, Yang Feng · 29. Dezember 2025

  Unsupervised learning has been widely used in many real-world applications. One of the simplest and most important unsupervised learning models is the Gaussian mixture model (GMM). In this work, we study the multi-task learning problem on GMMs, which aims to leverage potentially similar GMM paramete…

• A Unification of Discrete, Gaussian, and Simplicial Diffusion

  Nuria Alina Chandra, Yucen Lily Li, Alan N. Amin, Alex Ali, Joshua Rollins, Sebastian W. Ober, Aniruddh Raghu, Andrew Gordon Wilson · 19. Dezember 2025

  To model discrete sequences such as DNA, proteins, and language using diffusion, practitioners must choose between three major methods: diffusion in discrete space, Gaussian diffusion in Euclidean space, or diffusion on the simplex. Despite their shared goal, these models have disparate algorithms, …

• Sublinear Variational Optimization of Gaussian Mixture Models with Millions to Billions of Parameters

  Sebastian Salwig, Till Kahlke, Florian Hirschberger, Dennis Forster, J\"org L\"ucke · 12. Dezember 2025

  Gaussian Mixture Models (GMMs) range among the most frequently used models in machine learning. However, training large, general GMMs becomes computationally prohibitive for datasets that have many data points $N$ of high-dimensionality $D$. For GMMs with arbitrary covariances, we here derive a high…

• Diffusion Models for Wireless Communications

  Mehdi Letafati, Samad Ali, Matti Latva-aho · 10. Dezember 2025

  A comprehensive study on the applications of denoising diffusion models for wireless systems is provided. The article highlights the capabilities of diffusion models in learning complicated signal distributions, modeling wireless channels, and denoising and reconstructing distorted signals. First, f…

• A Particle Algorithm for Mean-Field Variational Inference

  Qiang Du, Kaizheng Wang, Edith Zhang, Chenyang Zhong · 9. Dezember 2025

  Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing mean-field variational inference (MFVI) is coordinate ascent variational inference…

• Discriminative classification with generative features: bridging Naive Bayes and logistic regression

  Zachary Terner, Alexander Petersen, Yuedong Wang · 2. Dezember 2025

  We introduce Smart Bayes, a new classification framework that bridges generative and discriminative modeling by integrating likelihood-ratio-based generative features into a logistic-regression-style discriminative classifier. From the generative perspective, Smart Bayes relaxes the fixed unit weigh…