Advanced Mathematical Modeling in Engineering

Latest papers

• Scaling and Distilling Text Embeddings for Better Diffusibility

  Zekai Zhang, Yunjie Tian, Yanjin He, Xiaoyan Zhang, Dongdi Zhao, Qing Qu, Di Fu · 2 October 2026

  Diffusion language models (DLMs) offer a promising alternative to autoregressive (AR) language generation. Recent advances in continuous DLMs, which apply latent diffusion to continuous text embeddings, raise a practical question: which embedding makes the best latent space, i.e., the most diffusibl…

• Hierarchical Continuous Diffusion Language Models

  Hui Ren, Zihan Li, Chang Liu, Huidong Liu, Alexander Schwing · 2 October 2026

  Discrete diffusion language models offer a compelling alternative to autoregressive generation for tasks demanding bidirectional reasoning and global constraint satisfaction. Yet they share a structural bottleneck: when decoding in parallel, each token is sampled independently from its marginal, sev…

• Discrete Wasserstein Flows for One-Step Generative Modeling

  Alessandro Micheli, Andrea Zerio, Samir Bhatt · 2 October 2026

  We introduce a new framework for one-step generative modelling on finite state spaces. To extend drifting beyond continuous domains, we use discrete Wasserstein geometry to define a target-relative KL gradient flow over the transitions of a reversible Markov kernel. We realize this probability flow …

• From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models

  Cristina L\'opez Amado, Marco Fumero, Francesco Locatello · 1 October 2026

  Diffusion models are typically viewed as stochastic processes that transform noise into data. We take a complementary perspective: a diffusion model defines a family of deterministic dynamical systems indexed by noise scale. At each fixed scale $\sigma$, we treat the denoiser as a self-map and study…

• Acceleration of Diffusion Language Model through Discrete Average Generator

  Yidong Ouyang, Zhengyan Wan, Themis Haris, Tian Tan, Liqian Peng, Henry Li, Ziqian Lin, Jianhang Chen, Maryam Karimzadehgan, Alec Go, George Michailidis · 1 October 2026

  Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few-step generation remains a fundamental challenge. In this work, we introduce the Discrete Average Generator, a principled extension of MeanFlow to Cont…

• Less Uniform Discrete Diffusion is More Powerful and Scalable

  Kaibo Wang, Ding Ding, Fangyu Ding, Zijin Feng, Han Shi, Haili Bai, Jiacheng Sun, Yang Xiang · 30 September 2026

  Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform training objective and condition-target confusion during sampling. To address these, we propose Less Uniform Diffusion (LUDI…

• Does Uniform Discrete Diffusion Need Time?

  Chunsan Hong, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Jong Chul Ye, Yuki Mitsufuji · 28 September 2026

  Uniform discrete diffusion models (UDMs) commonly use explicit time conditioning, but we find that it can often be unnecessary in practice. In this paper, we first show that the population-optimal UDM predictor generally depends on time: time controls how much the model should trust the observed con…

• Spectral Amplitude Purification in Distribution Matching for Diffusion Distillation

  Zhenyu Zhou, Can Wang, Chun Chen, Zeyu Zheng, Defang Chen · 25 September 2026

  Distribution Matching Distillation (DMD) enables high-quality diffusion sampling in only a few steps, but its optimization dynamics remain dominated by coarse, low-frequency signals, delaying the recovery of fine-grained details. We identify a pronounced concentration of spectral amplitudes at low f…

• Discrete Diffusion Models via Evolving Variational Autoregressive Networks

  Kewen Pan, Ying Tang · 24 September 2026

  Conventional score-based diffusion models learn scores without representing normalized densities, whereas tractable normalized models support both sampling and direct likelihood evaluation. A recent tensor-network approach provides such a representation but is largely restricted to low-dimensional l…

• Double Descent and Malign Overfitting in Diffusion Models

  Rapha\"el Urfin, Tony Bonnaire, Giulio Biroli, Marc M\'ezard · 23 September 2026

  Conventional wisdom in deep learning holds that overparameterization---having more parameters $p$ than training samples $n$---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might …

• Optimizers for Diffusion Models: A Controlled Benchmark

  Arman Bolatov, Egor Shulgin, David Li, Abduragim Shtanchaev, Sebastian U. Stich, Maxim Panov, Eric Moulines, Peter Richt\'arik, Martin Tak\'a\v{c} · 22 September 2026

  Discrete diffusion models now match autoregressive language models on several benchmarks, while the question of how best to train them has received far less attention: the optimizer is inherited from one paper to the next and never compared. New optimizers, meanwhile, are validated almost exclusivel…

• Leveraging Inference-Time Compute for Diffusion Models via Global Scheduling of Denoising Trajectories

  Yuan Cao, Yifu Tang, Hangqi Li, Zeyu Zheng · 22 September 2026

  Diffusion models generate a sample by traversing a denoising trajectory, a sequence of stochastic noise-reduction steps that transforms pure noise into a draw from a target distribution. At deployment time, additional computation can improve sample quality without retraining: at each step, the sampl…

• D-IMPL: A Diffusion-based Solver for Parameterized BBOs

  Yang Hu, Na Li · 22 September 2026

  Diffusion models have demonstrated strong power in generative modeling tasks across multiple domains, exhibiting a remarkable capability of learning complex distributions from samples. In this paper, we leverage such capability to design an efficient universal diffusion-based solver for parameterize…

• Schedule optimization for tau-leaping in masked discrete diffusion

  Cecilia Secchi, Giacomo Zanella · 21 September 2026

  Masked discrete diffusion models are commonly accelerated using the so-called tau-leaping discretization method, which reveals several coordinates in parallel at each sampling step. The sampler replaces the joint conditional law of each revealed block by a product distribution, incurring a factoriza…

• Parallelism, critical windows, and separations among diffusion language models

  Sitan Chen, Liye Wang · 18 September 2026

  A popular selling point of diffusion large language models (dLLMs) is their capacity for parallelism: the ability to generate sequences of text far more efficiently than autoregressive models, which require one forward pass per token. Yet among the many competing paradigms for dLLMs, from masked to …

• Limits of Confidence in Diffusion

  Russ Webb, Amitis Shidani, Alice Bizeul, Dan Busbridge · 18 September 2026

  Discrete diffusion, including remasking and uniform-state samplers, generate a sequence by writing multiple token positions per step, drawing each from a per-position distribution and choosing which positions to write from those same distributions. For domains of general interest (pixels, phonemes, …

• Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning

  Sophia Tang, Shiyi Wang · 16 September 2026

  Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipe…

• Diffusion Models and Concept Formation

  Zekun Wang, Karthik Singaravadivelan, Christopher J. MacLellan · 14 September 2026

  Humans organize knowledge into a taxonomy of concepts with nested levels of abstraction and a \emph{basic level} at which people recognize and name objects with the least cognitive effort. Cobweb is a classic cognitive account of this ability, an incremental learner that builds a probabilistic conce…

• Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

  Luyi Jia, Boyan Zhang, Yilun Liu, Steffen Rulands · 11 September 2026

  Diffusion and flow-matching schedules control the signal and noise coefficients that mix data and noise along affine probability paths. Minimizing a kinetic action defined on coefficient paths, motivated by optimal transport, helps explain strong baselines but remains model-agnostic and ignores pred…

• Let It Go or Learn to Self-Correct: Continuous Diffusion for Constrained Discrete Tasks

  Mariia Drozdova, St\'ephane Liem Nguyen, Fran\c{c}ois Fleuret · 9 September 2026

  Denoising Diffusion Probabilistic Models (DDPMs) generate samples by starting from noise and repeatedly denoising while keeping each update close to the current noisy state. This behavior is effective in many continuous domains, but its role is less clear for globally constrained discrete tasks, suc…

• Minimax Lower Bound for Estimating Diffusion-based Local Intrinsic Dimension

  Jaehee Seo, Wontae Jeong, Jisu Kim · 7 September 2026

  While diffusion-based methods have recently emerged as effective tools for probing the intrinsic geometry of high-dimensional data, their statistical difficulty remains largely unexplored. We study estimation of the finite-scale population functional underlying FLIPD (Kamkari et al., 2024; arXiv:240…

• Conditioning Degenerate Diffusion Models

  U\u{g}ur Ayd{\i}n, Tamer Ba\c{s}ar · 4 September 2026

  Current conditioned generative models heavily rely on score functions for guidance during training. When the generative model is a diffusion process with a singular diffusion coefficient and the underlying (conditional) densities either do not exist or are not smooth, we use causal optimal transport…

• Diffusion as a Training Curriculum for Timestep-Free Iterative Reasoning

  Mariia Drozdova, Aidan Sirbu, Pietro Miotti, Robert Obryk, Mayalen Etcheverry, Eyvind Niklasson, Blake Richards · 2 September 2026

  Diffusion models and recursive reasoners are both iterative, but they carry information across iterations differently. We add a persistent hidden state to a diffusion denoiser and remove its timestep conditioning, leaving a single shared update that can be run to arbitrary depth. The result is an an…

• Diffusion Based Unpaired Data Learning for Inverse Problems

  Chenglong Bao, Yiming Dang, Chenguang Duan, Yuling Jiao, Defeng Sun · 2 September 2026

  Data is important in many deep learning-based inverse problem solvers. However, obtaining sufficient paired data in many scenarios remains highly challenging, while unpaired data is cheap. To maximize data utilization, this paper proposes LUD-DIF, a diffusion-based approach for solving inverse probl…

• ASSERT: Adaptive Stochastic Sampling for Robust Diffusion Models on Analog Compute-in-Memory Hardware

  Yuannuo Feng, Yizhe Chen, Wenshuai Yao, Yuxin Xie, Ngai Wong, Wenyong Zhou, Wang Kang · 2 September 2026

  Diffusion models achieve strong image generation quality but incur high iterative denoising costs. Analog compute-in-memory (CIM) can accelerate matrix-vector multiplications, yet spatial memory variations perturb weights and accumulate during sampling. Unlike conventional neural networks, diffusion…