Natural Language Processing Techniques

Latest papers

• Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models

  O. Ibrahimzade, K. Tabasaransky · 10 April 2026

  Large language models (LLMs) have transformed natural language processing, yet their capabilities remain uneven across languages. Most multilingual models are trained primarily on high-resource languages, leaving many languages with large speaker populations underrepresented in both training data an…

• AV-SQL: Decomposing Complex Text-to-SQL Queries with Agentic Views

  Minh Tam Pham, Trinh Pham, Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Thanh Tam Nguyen · 10 April 2026

  Text-to-SQL is the task of translating natural language queries into executable SQL for a given database, enabling non-expert users to access structured data without writing SQL manually. Despite rapid advances driven by large language models (LLMs), existing approaches still struggle with complex q…

• Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction

  Jackson Petty, Jaulie Goe, Tal Linzen · 10 April 2026

  Low-resource languages pose a challenge for machine translation with large language models (LLMs), which require large amounts of training data. One potential way to circumvent this data dependence is to rely on LLMs' ability to use in-context descriptions of languages, like textbooks and dictionari…

• AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models

  Wenyu Li, Xiaoqi Jiao, Yi Chang, Guangyan Zhang, Yiwen Guo · 10 April 2026

  The creation of high-quality multimodal datasets remains fundamental for advancing role-playing capabilities in large language models (LLMs). While existing works predominantly focus on text-based persona simulation, Audio Role-Playing (ARP) presents unique challenges due to the need for synchronize…

• How to Evaluate Speech Translation with Source-Aware Neural MT Metrics

  Mauro Cettolo, Marco Gaido, Matteo Negri, Sara Papi, Luisa Bentivogli · 10 April 2026

  Automatic evaluation of ST systems is typically performed by comparing translation hypotheses with one or more reference translations. While effective to some extent, this approach inherits the limitation of reference-based evaluation that ignores valuable information from the source input. In MT, r…

• Distributed Interpretability and Control for Large Language Models

  Dev Arpan Desai, Shaoyi Huang, Zining Zhu · 9 April 2026

  Large language models that require multiple GPU cards to host are usually the most capable models. It is necessary to understand and steer these models, but the current technologies do not support the interpretability and steering of these models in the multi-GPU setting as well as the single-GPU se…

• ADOPT: Adaptive Dependency-Guided Joint Prompt Optimization for Multi-Step LLM Pipelines

  Minjun Zhao, Xinyu Zhang, Shuai Zhang, Deyang Li, Ruifeng Shi · 9 April 2026

  Multi-step LLM pipelines can solve complex tasks, but jointly optimizing prompts across steps remains challenging due to missing step-level supervision and inter-step dependency. We propose ADOPT, an adaptive dependency-guided joint prompt optimization framework for multi-step LLM pipelines. ADOPT a…

• Detecting and Explaining (In-)equivalence of Context-Free Grammars

  Marko Schmellenkamp, Thomas Zeume, Sven Argo, Sandra Kiefer, Cedric Siems, Fynn Stebel · 9 April 2026

  We propose a scalable framework for deciding, proving, and explaining (in-)equivalence of context-free grammars. We present an implementation of the framework and evaluate it on large data sets collected within educational support systems. Even though the equivalence problem for context-free languag…

• TransAgent: Enhancing LLM-Based Code Translation via Fine-Grained Execution Alignment

  Zhiqiang Yuan, Weitong Chen, Hanlin Wang, Xin Peng, Zhenpeng Chen, Yiling Lou · 8 April 2026

  Code translation transforms code between programming languages while preserving functionality, which is critical in software development and maintenance. While traditional learning-based code translation methods have limited effectiveness due to the lack of sufficient parallel training data, Large L…

• On the Geometry of Positional Encodings in Transformers

  Giansalvo Cirrincione · 8 April 2026

  Neural language models process sequences of words, but the mathematical operations inside them are insensitive to the order in which words appear. Positional encodings are the component added to remedy this. Despite their importance, positional encodings have been designed largely by trial and error…

• Unlocking the Potential of Diffusion Language Models through Template Infilling

  Junhoo Lee, Seungyeon Kim, Nojun Kwak · 8 April 2026

  Diffusion Language Models (DLMs) have emerged as a promising alternative to Autoregressive Language Models, yet their inference strategies remain limited to prefix-based prompting inherited from the autoregressive paradigm. In this paper, we propose Template Infilling (TI), a tailored conditioning m…

• LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification

  Penghui Yang, Cunxiao Du, Fengzhuo Zhang, Haonan Wang, Tianyu Pang, Chao Du, Bo An · 8 April 2026

  As Large Language Models (LLMs) can now process extremely long contexts, efficient inference over these extended inputs has become increasingly important, especially for emerging applications like LLM agents that highly depend on this capability. Speculative decoding (SD) offers a promising lossless…

• Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling

  Yongchang Hao, Lili Mou · 8 April 2026

  Speculative sampling (SpS) has been successful in accelerating the decoding throughput of auto-regressive large language models by leveraging smaller draft models. SpS strictly enforces the generated distribution to match that of the verifier LLM. This is unnecessarily restrictive as slight variatio…

• JTON: A Token-Efficient JSON Superset with Zen Grid Tabular Encoding for Large Language Models

  Gowthamkumar Nandakishore · 8 April 2026

  When LLMs process structured data, the serialization format directly affects cost and context utilization. Standard JSON wastes tokens repeating key names in every row of a tabular array--overhead that scales linearly with row count. This paper presents JTON (JSON Tabular Object Notation), a strict …

• Phonetic Perturbations Reveal Tokenizer-Rooted Safety Gaps in LLMs

  Darpan Aswal, Siddharth D Jaiswal · 8 April 2026

  Safety-aligned LLMs remain vulnerable to digital phenomena like textese that introduce non-canonical perturbations to words but preserve the phonetics. We introduce CMP-RT (code-mixed phonetic perturbations for red-teaming), a novel diagnostic probe that pinpoints tokenization as the root cause of t…

• LPC-SM: Local Predictive Coding and Sparse Memory for Long-Context Language Modeling

  Keqin Xie · 7 April 2026

  Most current long-context language models still rely on attention to handle both local interaction and long-range state, which leaves relatively little room to test alternative decompositions of sequence modeling. We propose LPC-SM, a hybrid autoregressive architecture that separates local attention…

• Beyond Linear Steering: Unified Multi-Attribute Control for Language Models

  Narmeen Oozeer, Luke Marks, Shreyans Jain, Fazl Barez, Amirali Abdullah · 7 April 2026

  Controlling multiple behavioral attributes in large language models (LLMs) at inference time is a challenging problem due to interference between attributes and the limitations of linear steering methods, which assume additive behavior in activation space and require per-attribute tuning. We introdu…

• SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression

  Xinhao Huang, You-Liang Huang, Zeyi Wen · 7 April 2026

  Large language models (LLMs) have demonstrated impressive capabilities across various tasks, but the billion-scale parameters pose deployment challenges. Although existing methods attempt to reduce the scale of LLMs, they require either special hardware support or expensive post-training to maintain…

• Many Preferences, Few Policies: Towards Scalable Language Model Personalization

  Cheol Woo Kum, Jai Moondra, Roozbeh Nahavandi, Andrew Perrault, Milind Tambe, Swati Gupta · 7 April 2026

  The holy grail of LLM personalization is a single LLM for each user, perfectly aligned with that user's preferences. However, maintaining a separate LLM per user is impractical due to constraints on compute, memory, and system complexity. We address this challenge by developing a principled method f…

• BiST: A Gold Standard Bangla-English Bilingual Corpus for Sentence Structure and Tense Classification with Inter-Annotator Agreement

  Abdullah Al Shafi, Swapnil Kundu Argha, M. A. Moyeen, Abdul Muntakim, Shoumik Barman Polok · 7 April 2026

  High-quality bilingual resources remain a critical bottleneck for advancing multilingual NLP in low-resource settings, particularly for Bangla. To mitigate this gap, we introduce BiST, a rigorously curated Bangla-English corpus for sentence-level grammatical classification, annotated across two fund…

• Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

  Serena Liu, Yutong Yang, Prisha Sheth, Weixuan Dong, Mingjiao Diao, Xinru Zhu, Nikhil Banga, Oscar Melendez, Arnav Sharma, Minda Zhao, Marina Lin, Mengyu Wang · 7 April 2026

  Large language models (LLMs) are used globally, and because much of their training data is in English, they typically perform best on English inputs. As a result, many non-native English speakers interact with them in English as a second language (ESL), and these inputs often contain typographical e…

• Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

  Qintong Zhang, Bin Wang, Victor Shea-Jay Huang, Junyuan Zhang, Zhengren Wang, Hao Liang, Conghui He, Wentao Zhang · 7 April 2026

  Document parsing (DP) transforms unstructured or semi-structured documents into structured, machine-readable representations, enabling downstream applications such as knowledge base construction and retrieval-augmented generation (RAG). This survey provides a comprehensive and timely review of docum…

• Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference

  Cornelius Kummer, Lena Jurkschat, Michael F\"arber, Sahar Vahdati · 6 April 2026

  With the wide adoption of language models for IR -- and specifically RAG systems -- the latency of the underlying LLM becomes a crucial bottleneck, since the long contexts of retrieved passages lead large prompts and therefore, compute increase. Prompt compression, which reduces the size of input pr…

• Fast NF4 Dequantization Kernels for Large Language Model Inference

  Xiangbo Qi, Chaoyi Jiang, Murali Annavaram · 6 April 2026

  Large language models (LLMs) have grown beyond the memory capacity of single GPU devices, necessitating quantization techniques for practical deployment. While NF4 (4-bit NormalFloat) quantization enables 4$\times$ memory reduction, inference on current NVIDIA GPUs (e.g., Ampere A100) requires expen…

• gen2seg: Generative Models Enable Generalizable Instance Segmentation

  Om Khangaonkar, Hamed Pirsiavash · 6 April 2026

  By pretraining to synthesize coherent images from perturbed inputs, generative models inherently learn to understand object boundaries and scene compositions. How can we repurpose these generative representations for general-purpose perceptual organization? We finetune Stable Diffusion and MAE (enco…