Error Correcting Code Techniques

Latest papers

• Twist, Don't Tilt: Trajectory-Exact Constrained Decoding for Masked Diffusion Models

  Aditya Thimmaiah, Lara Marinov, Jayanth Srinivasa, Haris Vikalo, Junyi Jessy Li, Milos Gligoric · 30 September 2026

  Constrained decoding for Masked Diffusion Language Models (MDLMs) aims to ensure that generated outputs satisfy a specified structure or syntax constraint. MDLMs generate outputs by repeatedly unmasking masked positions present in their current state. Recent strategies for constrained decoding const…

• Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling

  Lixuan Wei, Wei Zhou, Jianwen Wu, Yipeng Shen, Meiling Wang, Haoran You · 16 September 2026

  Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative unmasking. However, dLLMs typically require many steps before token confidence reaches the decoding threshold, resulting in inefficient inference even…

• MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes

  Jonggyu Jang, Hongjae Nam, Vishrant Tripathi, David J. Love, Hyun Jong Yang · 2 September 2026

  Feedback-based coding schemes have demonstrated substantial performance gains over today's open-loop coding schemes. Unfortunately, these gains are usually achieved in idealized settings with perfect feedback. Over the last few years, machine learning-based schemes have been shown to be promising so…

• Multi-agent discovery of practical quantum LDPC codes

  Dongheng Qian, Tianyi Li · 11 August 2026

  Quantum low-density parity-check (qLDPC) codes can encode multiple logical qubits using sparse parity checks, yet searching for useful finite-length instances remains a challenging design problem because code performance must be optimized while satisfying practical constraints. Motivated by recent a…

• Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware

  Param Chordiya · 21 July 2026

  Single-stream autoregressive decoding of large language models is bound by memory bandwidth: each generated token requires one full forward pass through the target model, and successive passes cannot be parallelized. Speculative decoding restructures this computation: a small draft model proposes $K…

• Computing with Stochastic Oracles in AI-Augmented Computation

  Jie Wang · 9 July 2026

  The Stochastic-Oracle Turing Machine (SOTM) framework models AI-augmented computation as the interaction of a probabilistic Turing machine with an oracle whose responses are drawn from context-dependent distributions. This paper studies what an SOTM can achieve under two oracle-response schemes: in …

• Deterministic Envelopes for Tamed SGLD: Decoupling Stochastic Gradient Noise and Localizing Taming

  Yiwei Zhou, Ziheng Chen · 7 July 2026

  Stochastic gradient Langevin algorithms often use tamed denominators to stabilize superlinear drifts. This paper shows that when the denominator depends on the current stochastic gradient, the transformed update can have a biased conditional mean even if the original stochastic gradient is unbiased.…

• HyperDFlash: Hyper-Connection-Aligned Block Speculative Decoding with Gated Residual Reduction

  Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmin Chen, Songwei Liu · 30 June 2026

  We present HyperDFlash, a block-parallel speculative decoding framework tailored to DeepSeek-V4's Hyper-Connections (HC). Despite the strong performance of DeepSeek-V4's native Multi-Token Prediction (MTP) module on initial token drafting, its draft accuracy degrades sharply at later positions, as e…

• Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization

  Rahid Zahid Alekberli, Hikmat Karimov · 30 June 2026

  We study the spectral perturbation of the empirical Fisher Information Matrix (FIM) of a parametric statistical model under two structured perturbations: departure of the input from a reference (in-distribution) ensemble, and finite-precision (quantized) perturbation of the model's parameters. For t…

• HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction

  Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmin Chen, Songwei Liu · 26 June 2026

  We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token Prediction (MTP) module in DeepSeek-V4, its draft accuracy …

• What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy

  Truong Xuan Khanh · 18 June 2026

  Grokking, the delayed jump from memorization to generalization, is usually tied to the weight norm: a smaller norm generalizes sooner. We ask what the norm actually controls. Holding the weight norm fixed by clamping and varying only an output temperature, we slide the grokking delay across its enti…

• Score Based Error Correcting Code Decoder

  Alon Helvits, Eliya Nachmani · 28 May 2026

  Error-correcting codes enable reliable communication, yet practical soft decoding remains challenging across code families and block lengths. We propose SB-ECC, a score-based decoder that casts decoding as continuous-time denoising. A neural denoiser defines a probability-flow ordinary differential …

• Analytical Correction for Subsampling Bias in Drifting Models

  Jiaru Zhang, Zeyun Deng, Juanwu Lu, Ziran Wang, Ruqi Zhang · 1 May 2026

  Drifting models are capable one-step generative models trained to follow a drifting field. The field combines attractive and repulsive softmax-weighted centroids over the data and current-generator distributions. In practice, only a minibatch of $n$ samples from each distribution is available, and e…

• VT-Former: Efffcient Transformer-based Decoder for Varshamov-Tenengolts Codes

  Yali Wei, Alan J. X. Guo, Zihui Yan, Yufan Dai, Wenjia Fan · 2 April 2026

  In recent years, widespread attention has been drawn to the challenge of correcting insertion, deletion, and substitution (IDS) errors in DNA-based data storage. Among various IDS-correcting codes, Varshamov-Tenengolts (VT) codes, originally designed for single-error correction, have been establishe…

• Greedy Multi-Path Block Verification for Faster Decoding in Speculative Sampling

  Rahul Thomas, Arka Pal · 20 February 2026

  The goal of $L$-step speculative decoding is to accelerate autoregressive decoding of a target model by using a cheaper draft model to generate a candidate path of $L$ tokens. Based on a verification algorithm involving target and draft model probabilities, a prefix of the candidate sequence is acce…

• Cross-Attention Speculative Decoding

  Wei Zhong, Manasa Bharadwaj, Yixiao Wang, Yipeng Ji, Chul Lee · 12 February 2026

  Speculative decoding (SD) is a widely adopted approach for accelerating inference in large language models (LLMs), particularly when the draft and target models are well aligned. However, state-of-the-art SD methods typically rely on tightly coupled, self-attention-based Transformer decoders, often …

• Neural Window Decoder for SC-LDPC Codes

  Dae-Young Yun, Hee-Youl Kwak, Yongjune Kim, Sang-Hyo Kim, Jong-Seon No · 30 January 2026

  In this paper, we propose a neural window decoder (NWD) for spatially coupled low-density parity-check (SC-LDPC) codes. The proposed NWD retains the conventional window decoder (WD) process but incorporates trainable neural weights. To train the weights of NWD, we introduce two novel training strate…

• A Reinforcement Learning Based Universal Sequence Design for Polar Codes

  David Kin Wai Ho, Arman Fazeli, Mohamad M. Mansour, Louay M. A. Jalloul · 29 January 2026

  To advance Polar code design for 6G applications, we develop a reinforcement learning-based universal sequence design framework that is extensible and adaptable to diverse channel conditions and decoding strategies. Crucially, our method scales to code lengths up to $2048$, making it suitable for us…

• TabDPT: Scaling Tabular Foundation Models on Real Data

  Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Alex Labach, Hamidreza Kamkari, Jesse C. Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L. Caterini, Maksims Volkovs · 21 January 2026

  Tabular data is one of the most ubiquitous sources of information worldwide, spanning a wide variety of domains. This inherent heterogeneity has slowed the development of Tabular Foundation Models (TFMs) capable of fast generalization to unseen datasets. In-Context Learning (ICL) has recently emerge…

• A Global Geometric Analysis of Maximal Coding Rate Reduction

  Peng Wang, Huikang Liu, Druv Pai, Yaodong Yu, Zhihui Zhu, Qing Qu, Yi Ma · 17 November 2025

  The maximal coding rate reduction (MCR$^2$) objective for learning structured and compact deep representations is drawing increasing attention, especially after its recent usage in the derivation of fully explainable and highly effective deep network architectures. However, it lacks a complete theor…

• Scalable Neural Decoders for Practical Real-Time Quantum Error Correction

  Changwon Lee, Tak Hur, Daniel K. Park · 28 October 2025

  Real-time, scalable, and accurate decoding is a critical component for realizing a fault-tolerant quantum computer. While Transformer-based neural decoders such as \textit{AlphaQubit} have demonstrated high accuracy, the computational complexity of their core attention mechanism, which scales as $\m…