Search: diffusion models

More than 1,000 papers match: here are the 1,000 most recent, ranked by relevance.

• Few-Step Diffusion Language Models via Trajectory Self-Distillation

  Tunyu Zhang, Xinxi Zhang, Ligong Han, Haizhou Shi, Xiaoxiao He, Zhuowei Li, Hao Wang, Kai Xu, Akash Srivastava, Chengzhi Mao, Hao Wang, Vladimir Pavlovic, Dimitris N. Metaxas · 18 May 2026 · Large Language Models

  Diffusion large language models (DLLMs) have emerged as powerful generative models with the promise of fast text generation through parallel decoding. However, realizing this potential in practice remains challenging: reducing the number of decoding steps, typically causes a substantial degradation …

• Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding

  Xun Fang, Yunchen Li, Hang Yuan, Zhou Yu · 15 May 2026 · Authorship Attribution and Profiling

  Discrete diffusion language models improve generation efficiency through parallel token prediction, but standard $X_0$ prediction methods introduce factorization errors by approximating the clean token posterior with independent token-wise distributions. This paper proposes Factorization-Error-Free …

• Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models

  Saba Ahmadi, Prasanna Parthasarathi, Yufei Cui · 15 May 2026 · Large Language Models

  Diffusion language models are a promising alternative to autoregressive models, yet post-training methods for them largely adapt reward-maximizing objectives. We identify a central failure mode in this setting we call trajectory locking: sampled reward-driven updates over-concentrate probability mas…

• Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models

  Yejin Lee, Yo-Sub Han · 14 May 2026 · Large Language Models

  Diffusion Language Models (DLMs) provide a promising alternative to autoregressive language models by generating text through iterative denoising and bidirectional refinement. However, this iterative generation paradigm also introduces unique safety vulnerabilities when harmful tokens generated at i…

• JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learning

  Jing Yu Lim, Rushi Shah, Zarif Ikram, Samson Yu, Haozhe Ma, Tze-Yun Leong, Dianbo Liu · 14 May 2026 · Reinforcement Learning in Robotics

  Diffusion world models have recently become competitive for online model-based reinforcement learning, but current approaches expose a tension: pixel diffusion is effective but computationally expensive while the latest latent diffusion approach improves efficiency yet performs subpar. The latter al…

• Entropy Aware Reward Guidance for Diffusion Language Model Alignment

  Atula Tejaswi, Litu Rout, Constantine Caramanis, Sanjay Shakkottai, Sujay Sanghavi · 14 May 2026 · Large Language Models

  Reward guidance, also known as posterior sampling, is a popular method for test-time adaptation and post-training in continuous diffusion models. In this paper, we study reward guidance for discrete diffusion language models; now, one cannot differentiate through the natural outputs of the model bec…

• Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models

  Omer Luxembourg, Haim Permuter, Eliya Nachmani · 14 May 2026 · Natural Language Processing Techniques

  Masked diffusion language models (MDLMs) promise fast, non-autoregressive text generation, yet existing samplers, which pick tokens to unmask based on model confidence, ignore interactions when unmasking multiple positions in parallel and effectively reduce to slow, autoregressive behavior. We propo…

• Understanding and Accelerating the Training of Masked Diffusion Language Models

  Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji, Seungryong Kim, Jong Chul Ye · 14 May 2026 · Generative Adversarial Networks and Image Synthesis

  Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more slowly than ARMs, which may become problematic when scaling MDMs to larger models. Therefore, we ask the following questio…

• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner

  Cai Zhou, Chenxiao Yang, Yi Hu, Chenyu Wang, Chubin Zhang, Muhan Zhang, Lester Mackey, Tommi Jaakkola, Stephen Bates, Dinghuai Zhang · 13 May 2026 · Large Language Models

  Diffusion language models, especially masked discrete diffusion models, have achieved great success recently. While there are some theoretical and primary empirical results showing the advantages of latent reasoning with looped transformers or continuous chain-of-thoughts, continuous diffusion model…

• Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models

  Hanhan Zhou, Shamik Roy, Rashmi Gangadharaiah · 13 May 2026 · Large Language Models

  Discrete diffusion language models (DLMs) generate text by iteratively denoising all positions in parallel, offering an alternative to autoregressive models. Controlled generation methods for DLMs, imported from autoregressive models, apply uniform intervention at every denoising steps. We show this…

• Relative Score Policy Optimization for Diffusion Language Models

  Zichao Yu, Shengze Xu, Bingqing Jiang, Wenyi Zhang, Difan Zou · 12 May 2026 · Large Language Models

  Diffusion large language models (dLLMs) offer a promising route to parallel and efficient text generation, but improving their reasoning ability requires effective post-training. Reinforcement learning with verifiable rewards (RLVR) is a natural choice for this purpose, yet its application to dLLMs …

• Edit-Based Refinement for Parallel Masked Diffusion Language Models

  Houxing Ren, Mingjie Zhan, Zimu Lu, Ke Wang, Yunqiao Yang, Haotian Hou, Junting Pan, Hongsheng Li · 12 May 2026 · Large Language Models

  Masked diffusion language models enable parallel token generation and offer improved decoding efficiency over autoregressive models. However, their performance degrades significantly when generating multiple tokens simultaneously, due to a mismatch between token-level training objectives and joint s…

• DARE: Diffusion Language Model Activation Reuse for Efficient Inference

  Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang, Chi-Chih Chang, Mohamed S. Abdelfattah, Diana Marculescu · 12 May 2026 · Large Language Models

  Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to auto-regressive (AR) models, offering greater expressive capacity and potential for parallel generation and faster inference. However, open-source dLLMs remain immature, lagging behind AR models in both efficiency and…

• How to Train Your Latent Diffusion Language Model Jointly With the Latent Space

  Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov, Nikita Gushchin, Ilya Koziev, Alexander Korotin, Dmitry Vetrov · 11 May 2026 · Large Language Models

  Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising entire sequences in parallel. The major challenge in latent diffusion modeling is constructing a suitable latent space. I…

• Guidance Is Not a Hyperparameter: Learning Dynamic Control in Diffusion Language Models

  Fan Zhou, Tim Van de Cruys · 11 May 2026 · Reinforcement Learning in Robotics

  Classifier-Free Guidance (CFG) is a widely used mechanism for controlling diffusion-based generative models, yet its guidance scale is typically treated as a fixed hyperparameter throughout generation. This static design yields a suboptimal controllability and quality tradeoff, as the optimal degree…

• DiffRetriever: Parallel Representative Tokens for Retrieval with Diffusion Language Models

  Shuai Wang, Yu Yin, Shengyao Zhuang, Bevan Koopman, Guido Zuccon · 11 May 2026 · Language and cultural evolution

  This paper shows how diffusion language models (DLMs) can be used as effective and efficient retrievers. Existing DLM-based retrievers (e.g., DiffEmbed) follow BERT-style encoding, representing each query or passage as a single mean-pooled vector. This ignores how DLMs are trained to generate respon…

• Continuous Latent Diffusion Language Model

  Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, Rui Zhu, Qiushan Guo, Feng Wang, Tao Yang, Hengshuang Zhao, Guoqiang Wei, Yan Zeng · 8 May 2026 · Large Language Models

  Large language models have achieved remarkable success under the autoregressive paradigm, yet high-quality text generation need not be tied to a fixed left-to-right order. Existing alternatives still struggle to jointly achieve generation efficiency, scalable representation learning, and effective g…

• Chainwash: Multi-Step Rewriting Attacks on Diffusion Language Model Watermarks

  Mohd Ruhul Ameen, Akif Islam, Nadim Mahmud, Md. Ekramul Hamid · 8 May 2026 · Handwritten Text Recognition Techniques

  Statistical watermarking is a common approach for verifying whether text was written by a language model. Most existing schemes assume autoregressive generation, where tokens are produced left to right and contextual hashing is well defined. Diffusion language models generate text by denoising token…

• Layer Collapse in Diffusion Language Models

  Alexander Conzelmann, Albert Catalan-Tatjer, Shiwei Liu · 8 May 2026 · Large Language Models

  Diffusion language models (DLMs) have recently emerged as competitive alternatives to autoregressive (AR) language models, yet differences in their activation dynamics remain poorly understood. We characterize these dynamics in LLaDA-8B and identify a striking layer-collapse property: a few early la…

• Consistent Diffusion Language Models

  Hasan Amin, Yuan Gao, Yaser Souri, Subhojit Som, Ming Yin, Rajiv Khanna, Xia Song · 4 May 2026 · Large Language Models

  Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps. In continuous domains, consistency training along th…

• ComboStoc: Combinatorial Stochasticity for Diffusion Generative Models

  Rui Xu, Jiepeng Wang, Hao Pan, Yang Liu, Xin Tong, Shiqing Xin, Changhe Tu, Taku Komura, Wenping Wang · 30 April 2026 · Business Strategy and Innovation

  In this paper, we study an under-explored but important factor of diffusion generative models, i.e., the combinatorial complexity. Data samples are generally high-dimensional, and for various structured generation tasks, additional attributes are combined to associate with data samples. We show that…

• Exploring Time Conditioning in Diffusion Generative Models from Disjoint Noisy Data Manifolds

  Liuzhuozheng Li, Zhiyuan Zhan, Shuhong Liu, Dengyang Jiang, Zanyi Wang, Guang Dai, Jingdong Wang, Mengmeng Wang · 29 April 2026 · Generative Adversarial Networks and Image Synthesis

  Practically, training diffusion models typically requires explicit time conditioning to guide the network through the denoising sampling process. Especially in deterministic methods like DDIM, the absence of time conditioning leads to significant performance degradation. However, other deterministic…

• On the Trainability of Masked Diffusion Language Models via Blockwise Locality

  Yuxiang Wang, Yu Xiang, Baojian Zhou, Qifang Zhao, Keyue Jiang, Yanghua Xiao, Xiaoxiao Xu · 29 April 2026 · Large Language Models

  Masked diffusion language models (MDMs) have recently emerged as a promising alternative to standard autoregressive large language models (AR-LLMs), yet their optimization can be substantially less stable. We study blockwise MDMs and compare them with AR-LLMs on three controlled tasks that stress di…

• Measuring Temporal Linguistic Emergence in Diffusion Language Models

  Harry Lu · 28 April 2026 · Language and cultural evolution

  Diffusion language models expose an explicit denoising trajectory, making it possible to ask when different kinds of information become measurable during generation. We study three independent 32-step runs of LLaDA-8B-Base on masked WikiText-103 text, each with 1{,}000 probe-training sequences and 2…

• DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models

  Dake Bu, Wei Huang, Andi Han, Hau-San Wong, Qingfu Zhang, Taiji Suzuki, Atsushi Nitanda · 28 April 2026 · Gene Regulatory Network Analysis

  Diffusion language models generate without a fixed left-to-right order, making token ordering a central algorithmic choice: which tokens should be revealed, retained, revised or verified at each step? Existing systems mainly use random masking or confidence-driven ordering. Random masking creates tr…