Natural Language Processing Techniques

Neueste Paper

• Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference

  Zifan He, Rui Ma, Yizhou Sun, Jason Cong · 2. Juni 2026

  Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation (RAG), and compressed contextual memory, to support complex reasoning. We show that these optimizations can be unified i…

• EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Models

  Hyundong Jin, Yo-Sub Han · 2. Juni 2026

  Controlling language model outputs is essential for ensuring structural validity, reliability, and downstream usability, and diffusion language models are no exception. Recent advances in diffusion language model decoding have extended output control beyond regular constraints to context-free gramma…

• Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction

  Yujia Tong, Yuxi Wang, Yunyang Wan, Tian Zhang, Junhao Dong, Jingling Yuan · 2. Juni 2026

  Model compression techniques such as quantization and pruning are widely used to reduce the deployment cost of large language models (LLMs), with existing evaluations focusing almost exclusively on accuracy preservation. However, in safety-critical applications, a model's ability to reliably quantif…

• Low-Resource Safety Failures Are Action Failures, Not Representation Failures

  Rashad Aziz, Ikhlasul Akmal Hanif, Fajri Koto · 2. Juni 2026

  Safety alignment learned in high-resource languages transfers poorly to low-resource languages. Models refuse harmful prompts in English but fail to refuse when the same prompts are translated into Swahili or Burmese. Adaptive steering methods like AdaSteer and CAST inherit this failure cross-lingua…

• CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards

  Wei Tian, Yuhao Zhou, Man Lan · 2. Juni 2026

  Large Language Model (LLM) based Chinese Grammatical Error Correction (CGEC) systems face two critical challenges: general-purpose models lack specialized linguistic priors for subtle grammatical distinctions, and Supervised Fine-Tuning (SFT) with Maximum Likelihood Estimation fails to optimize for …

• DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention

  Younjoo Lee, Seungkyun Dan, Junghoo Lee, Jaiyoung Park, Jung Ho Ahn · 2. Juni 2026

  Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We…

• T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Models

  Minki Kang, Jongwon Jeong, Jaewoong Cho · 2. Juni 2026

  Recent studies have demonstrated that test-time compute scaling effectively improves the performance of small language models (sLMs). However, prior research has mainly examined test-time compute scaling with an additional larger model as a verifier, leaving verification by sLMs underexplored. In th…

• From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression

  Elia Cunegatti, Marcus Vukojevic, Erik Nielsen, Giovanni Iacca · 2. Juni 2026

  Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules. Existing replacement-based methods share two design constraints: full-layer granularity and contiguous selection. We argue that this is overl…

• French parsing enhanced with a word clustering method based on a syntactic lexicon

  Anthony Sigogne, Matthieu Constant, Eric Laporte · 2. Juni 2026

  This article evaluates the integration of data extracted from a French syntactic lexicon, the Lexicon-Grammar (Gross, 1994), into a probabilistic parser. We show that by applying clustering methods on verbs of the French Treebank (Abeill\'e et al., 2003), we obtain accurate performances on French wi…

• d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation

  Guanghan Wang, Gilad Turok, Yair Schiff, Marianne Arriola, Volodymyr Kuleshov · 2. Juni 2026

  While diffusion language models (DLMs) have achieved competitive performance in text generation, improving their reasoning ability with reinforcement learning remains an active research area. Here, we introduce d2, a reasoning framework tailored for masked DLMs. Central to our framework is a new pol…

• EuroBERT: Scaling Multilingual Encoders for European Languages

  Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves, Andr\'e Martins, Ayoub Hammal, Caio Corro, C\'eline Hudelot, Emmanuel Malherbe, Etienne Malaboeuf, Fanny Jourdan, Gabriel Hautreux, Jo\~ao Alves, Kevin El Haddad, Manuel Faysse, Maxime Peyrard, Nuno M. Guerreiro, Patrick Fernandes, Ricardo Rei, Pierre Colombo · 2. Juni 2026

  General-purpose multilingual vector representations, used in retrieval, regression and classification, are traditionally obtained from bidirectional encoder models. Despite their wide applicability, encoders have been recently overshadowed by advances in generative decoder-only models. However, many…

• Pairwise Reference Alignment as a Model-Level Ordinal Observable

  Mujing Li · 1. Juni 2026

  Pairwise preference data is widely used in language-model evaluation and alignment, often for model ranking, reward modeling, or preference optimization. This note formulates a more basic measurement question: given a reference distribution of pairwise preferences, what model-level quantity is estim…

• Assign and Add: A Mechanistic Study of Compositional Arithmetic

  Brady Exoo, Alberto Bietti, John Sous · 1. Juni 2026

  Large language models are able to compose skills in order to perform complex tasks, many of which might not have been seen during training. The details of how exactly this composition occurs remain elusive. In this paper, we study a mechanism for compositional generalization in transformers by consi…

• Neuro-symbolic Syntactic Parsing: Shaping a Neural Network with the CYK Algorithm

  Fabio Massimo Zanzotto, Federico Ranaldi, Giorgio Satta · 1. Juni 2026

  In this paper, we show the possibility of a direct injection of algorithms into neural network architecture. We focus on a complex algorithm, that is, Cocke-Youger-Kasami (CYK) for parsing context-free grammars in Chomsky Normal Form and we propose CYKNN, a simple recurrent neural network architectu…

• Speculative Decoding Across Languages

  Nirajan Paudel, Michael Ginn, Luc De Nardi, Alexis Palmer · 1. Juni 2026

  Speculative decoding has become a crucial component of large language model (LLM) inference, enabling faster generation by drafting multiple tokens and verifying them in parallel. However, small draft models tend to suffer from disproportionately poor multilingual capabilities. Thus, when generating…

• Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions

  Wesley Scivetti, Melissa Torgbi, Austin Blodgett, Mollie Shichman, Taylor Hudson, Claire Bonial, Harish Tayyar Madabushi · 1. Juni 2026

  The web-scale of pretraining data has created an important evaluation challenge: to disentangle linguistic competence on cases well-represented in pretraining data from generalization to out-of-domain language, specifically the dynamic, real-world instances less common in pretraining data. To this e…

• Trust-Region Behavior Blending for On-Policy Distillation

  Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov, Nikita Balagansky, Boris Shaposhnikov, Daria Korotyshova, Daniil Gavrilov · 1. Juni 2026

  On-policy distillation (OPD) trains a student on prefixes sampled from its own policy while matching a stronger teacher. This addresses the prefix mismatch of offline distillation, but early student rollouts can still be poor, placing teacher supervision on weak or low-quality prefixes. We propose T…

• DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs

  Sara Papi, Luisa Bentivogli · 1. Juni 2026

  Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit align…

• XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks

  Purvam Jain, Preethi Jyothi, Vihari Piratla, Suvrat Raju · 1. Juni 2026

  We introduce a set of synthetic algorithmic tasks to detect cross-lingual gaps in the abilities of large language models. Our benchmark is commensurate across languages, since it requires models to perform the same underlying task in different languages; scalable, since each task can be generated at…

• ProofWala: A Framework for Multilingual Proof Data Synthesis and Theorem-Proving

  Amitayush Thakur, George Tsoukalas, Greg Durrett, Swarat Chaudhuri · 1. Juni 2026

  Neural approaches to theorem proving require robust infrastructure for interfacing with interactive theorem provers (ITPs), extracting structured proof data, and executing proof search at scale. However, existing tooling is often assistant-specific and oriented toward file-level execution, making re…

• Accelerating Constrained Decoding with Token Space Compression

  Michael Sullivan, Alexander Koller · 29. Mai 2026

  To guarantee that an LLM's outputs conform to a specified structure, context-free grammar (CFG) decoding engines force the selection of next tokens that produce strings that conform to a given CFG. While current CFG-constrained decoding engines are highly optimized, the inherent costs arising from t…

• Conformal Certification of Reasoning Trace Prefixes

  Matt Y. Cheung, Ashok Veeraraghavan, Hanjie Chen, Guha Balakrishnan · 29. Mai 2026

  Language model reasoning traces are rarely all-or-nothing; they frequently contain valid intermediate steps before a critical error occurs. Existing uncertainty quantification methods typically certify final answers or entire responses, failing to provide statistical guarantees for the proportion of…

• LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Models

  Qingyu Ren, Qianyu He, Jingwen Chang, Geng Zhang, Jiajie Zhu, Xingzhou Chen, Zhuofei Shi, Jiaqing Liang, Yanghua Xiao, Han Xia, Zeye Sun, Fei Yu · 29. Mai 2026

  Instruction following is critical for large language models, yet real-world instructions often involve multiple constraints with logical structures, such as parallel composition, sequential dependencies, and conditional branching. Existing methods typically construct data by simply combining constra…

• Less Is More: Elevating RAG via Performance-Driven Context Compression

  Ziqiang Cui, Yunpeng Weng, Xing Tang, Peiyang Liu, Shiwei Li, Bowei He, Jiamin Chen, Yansen Zhang, Xiuqiang He, Chen Ma · 29. Mai 2026

  Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for improving the timeliness of knowledge updates and the factual accuracy of large language models. However, incorporating a large volume of retrieved documents significantly increases input length, leading to prohibitive comp…

• Reasoning with Sampling: Cutting at Decision Points

  Felix Zhou, Anay Mehrotra, Quanquan C. Liu · 29. Mai 2026

  Frontier reasoning models are produced by posttraining base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power distribution, elicits comparable reasoning without additional…