Natural Language Processing Techniques

Derniers papiers

• Imbalanced Data Clustering via Targeted Data Augmentation Using GMM and LLM

  Noor Khalal, Abdallah Alaa-Eddine Djamai, Imed Keraghel, Mohamed Nadif · 3 août 2026

  In Natural Language Processing (NLP), dealing with underrepresented topics is challenging, especially in unsupervised tasks where clustering might not adequately capture minority topics. To tackle this challenge, our paper presents a novel unsupervised data augmentation method that integrates Gaussi…

• Cross-Lingual Transfer for Machine Translation in Turkic Languages

  Omer Burak Cinar, Mehmet Mert Dalkilic, Cagri Toraman · 3 août 2026

  Cross-lingual transfer is central to low-resource machine translation, but its behavior within closely related language families remains insufficiently characterized. We study transfer among five Turkic languages; Turkish, Azerbaijani, Uzbek, Kazakh, and Kyrgyz; using pairwise transfer matrices. In …

• Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation

  Yongshi Ye, Biao Fu, Chongxuan Huang, Yidong Chen, Xiaodong Shi · 3 août 2026

  Multi-domain machine translation (MDMT) poses a unique challenge due to varying levels of linguistic complexity across domains. Inspired by human translators' ability to adapt reasoning effort based on difficulty, we propose TwT (Translation with Thought), a resource-rational framework that learns t…

• PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction

  Pirzada Suhail, Nagasai Saketh Naidu, Atanu R Sinha, Amit Sethi · 3 août 2026

  Large language models (LLMs) generate text by auto-regressively sampling the next token. This inherently leads to a many-to-many mapping between prompts and responses, complicating the task of inferring prompts from observed outputs. Prior work on LLM inversion frames prompt recovery as a semantic r…

• Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models

  Xingyu Mou, Zijin Huang, Tianze Zhang, Yuxin Ma, Lanning Wei, Zengfeng Huang, Da Zheng, Lun Du · 30 juillet 2026

  Block diffusion is the dominant approach for scaling discrete diffusion language models (dLLMs), as fixed-size blocks preserve parallel decoding while keeping quadratic attention costs tractable. Yet blockwise generation creates a structural weakness: tokens near a block boundary lack future cross-b…

• Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training

  Suvadeep Hajra · 30 juillet 2026

  Autoregressive transformer language models frequently exhibit training instability when trained on long sequences, particularly under low-precision arithmetic. Although this instability is often accompanied by attention-logit explosion, its underlying cause remains poorly understood. In this work, w…

• Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities

  Zhiwei Hao, Jianyuan Guo, Li Shen, Yong Luo, Han Hu, Guoxia Wang, Dianhai Yu, Yonggang Wen, Dacheng Tao · 30 juillet 2026

  Large language models (LLMs) have achieved impressive performance across various domains. However, the substantial hardware resources required for their training present a significant barrier to efficiency and scalability. To mitigate this challenge, low-precision training techniques have been widel…

• On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora?

  Joachim Minder (ALTAE), Guillaume Wisniewski (LLF - UMR7110), Natalie K\"ubler (ALTAE) · 29 juillet 2026

  Specialised translation relies on the use of documentary and terminological resources, including corpora. These resources are particularly useful for terminology. However, their compilation and exploitation have several limitations: they require time, technical skills and access to data that can be …

• DocAnnot -- Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation

  Siddartha Reddy, Harikrishnan P M, Goutham Vignesh, Varun V, Vishal Vaddina · 29 juillet 2026

  Key Information Extraction (KIE) is vital for many document applications, but creating training datasets is traditionally a time-consuming manual process. We introduce DocAnnot, a framework that significantly accelerates KIE dataset generation. DocAnnot leverages a Large Vision Language Model (LVLM)…

• Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors

  Jianfei Ma, Zhaoxin Feng, Emmanuele Chersoni, Si Chen · 29 juillet 2026

  Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic re…

• Construction-Driven Injection: Linguistically-Grounded Edit-Based Code-Mixing Fingerprints for Large Language Models

  Yongyi Cui, Yue Li, Tianbao Jiang, Xin Yi · 29 juillet 2026

  Large language models (LLMs) are costly intellectual assets that remain exposed to unauthorized redistribution and commercial misuse. Injected fingerprints, i.e., trigger--target pairs embedded in model behavior, offer a practical, black-box-verifiable ownership signal, but existing methods decouple…

• Instruction-Tuned Language Models Cannot Sample from Distributions They Can Describe

  Chaemin Jang, Dongman Lee, Jihee Kim · 29 juillet 2026

  Silicon sampling uses language models as proxies for human survey respondents, treating each model call as an independent draw from the persona's response distribution. We show this draw does not exist: instruction-tuned models do not sample from distributions, they collapse to a single output. The …

• DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

  Tan T. Nguyen, Quan V. Dang · 28 juillet 2026

  As the inference phase of Large Language Models (LLMs) requires handling long context windows, the Key-Value (KV) cache initially appears to address this challenge but eventually becomes a significant bottleneck as the context window continues to grow. Low-rank compression has recently been studied …

• DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

  He Zhang · 28 juillet 2026

  The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data. Existing dynamic data scheduling methods face critical limitations in industrial-scale pretraining and supervised fine-tuning (SFT): data selection incurs prohibitive O…

• The Tokenizer Tax: Quantifying and Explaining the Cross-Lingual Cost of Subword Tokenization for Indian Languages

  Priyansh Srivastava · 28 juillet 2026

  Large language models (LLMs) process text through subword tokenizers rather than directly reading characters or words. Because these tokenizers are trained predominantly on English-centric corpora, they introduce a systematic and often overlooked disadvantage for many non-English languages. In this …

• BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis

  Poornima Kumaresan, Pavithra Muruganantham, Lakshmi Rajendran, Santhosh Sivasubramani · 28 juillet 2026

  Standard subword tokenization algorithms such as Byte-Pair Encoding (BPE) and SentencePiece are trained predominantly on modern language corpora and produce inefficient segmentations when applied to classical Indian languages. Sanskrit, Tamil, and other classical Indic languages exhibit agglutinativ…

• LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

  Junsung Hwang · 28 juillet 2026

  Serving large language models at long context is bottlenecked by the key-value (KV) cache, which is read in full at every decode step. Attention keys are locally low-rank though globally high-rank: shared low-rank bases discard page-specific directions that a page's own compact basis retains. LOCKS …

• AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems

  Nilesh Prasad Pandey, Jason Kong, Lanxiang Hu, Quanling Zhao, Yujie Zhao, Onat Gungor, Hao Zhang, Tajana Rosing · 27 juillet 2026

  Memory-augmented LLM agents maintain context across hundreds of interactions through agentic memory systems that actively curate retrieved content with LLM-generated metadata such as summaries, keywords, and tags. From an inference cost standpoint, every retrieval triggers a full re-encoding of thes…

• Tractable Hierarchical Control of Autoregressive Language Models

  Max Scribner, Antonio Vergari, Vaishak Belle · 24 juillet 2026

  Constraining the generation of autoregressive large language models (LLMs) is an important component of integrating language models into formal systems. In the generation of code and data for tasks like program synthesis, ensuring that language models produce syntactically valid output is a prerequi…

• Test-Time Scaling via Error Localization

  Rajiv Shailesh Chitale, Rahul Madhavan, Taneesh Gupta, Deepanway Ghosal, Aravindan Raghuveer · 24 juillet 2026

  Scaling inference-time computation has emerged as a reliable method to improve the performance of large language models on complex reasoning and programming tasks. However, standard approaches such as independent sampling and sequential multi-turn refinement operate without token-level credit assign…

• Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing

  Jea Kwon, Jiwon Kim, Dong-kyum Kim, Meeyoung Cha · 24 juillet 2026

  While language models remain frozen at their training state, the world evolves continuously. Knowledge editing has emerged as a key alternative to full retraining, but its deployment is bottlenecked by the erosion of core capabilities: mathematical and programmatic reasoning collapse while encyclope…

• SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification

  Pragaash Ponnusamy, Shivam Sahni, Jue Wang, Tri Dao · 24 juillet 2026

  Sampling in LLM inference comprises a combinatorial set of logit processing, token selection, and verification operations for speculative decoding. However, existing implementations either accelerate only subsets of this pipeline, rely on multiple kernel launches, or assume homogeneous sampling beha…

• GLAN-QnA-KR: A Seedless Taxonomy-Driven Korean Instruction Corpus

  Daekeun Kim · 24 juillet 2026

  We release GLAN-QnA-KR, a 303,581-row openly redistributable Korean instruction-QA corpus produced via the seedless taxonomy-driven GLAN synthesis pipeline with Microsoft's Phi-3.5-MoE-instruct as the producer model (generation: 2024-12; release: 2024-12; licence: OpenRAIL). The corpus spans a flat …

• Co-Evolving LLM Evaluators and Policies via DynamicRubric

  Beining Wang, Weihang Su, Hongtao Tian, Hao Kong, Tao Yang, Ting Yao, Qingyi Pan, Yueyue Wu, Qingyao Ai, Min Zhang, Yiqun Liu · 23 juillet 2026

  Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models. As policies improve, these sampled responses become close in quality. These close candidates create a bottleneck for policy optimization: collapsed relative evaluator scor…

• LatentMT: Machine Translation with Latent Reasoning

  Wei-Rui Chen, Samar M. Magdy, Chiyu Zhang, Wenhui Zhu, Zhipeng Wang, Muhammad Abdul-Mageed · 22 juillet 2026

  Latent-reasoning looped language models (LoopLMs) offer a different scaling path for machine translation (MT): instead of increasing parameter count or emitting explicit chain-of-thought tokens, they spend additional recurrent computation inside hidden states. We introduce LatentMT, the first system…