Natural Language Processing Techniques

Latest papers

• L20-Edu-135M: An Auditable Single-GPU Study of Data-Efficient Small Language Modeling

  Yin Li · 23 June 2026

  Small language models are cheap to serve and feasible on local hardware, but strong public 135M-class systems are commonly trained with hundreds of billions to trillions of tokens on large clusters. We study a sharply resource-constrained regime: a complete 134.5M-parameter language-model pipeline e…

• Context-Aware Distillation and Ablation for Text2DSL

  Alexander V. Kozachok, Alexander M. Nazimov, Shamil G. Magomedov · 23 June 2026

  We extend our prior work on Text2DSL automatic generation of domain-specific language (DSL) code from natural language descriptions along two complementary axes. First, we replace prompt-only synthetic generation with context-aware distillation, in which a teacher large language model (DeepSeek-V4-F…

• UniRank: Unified Rank Allocation for Low-Rank LLM Compression

  Chao Han, Haozhe Hu, Fei Ma, Wei Zhang, Xiaoyu Shen · 23 June 2026

  Low-rank decomposition serves as a promising compression paradigm for large language models, however, rank allocation remains challenging: manual rules lack generalizability, and learning-based approaches incur heavy computational overhead. To address these issues, we formulate global low-rank alloc…

• Generalization of Fine-Tuned Uncertainty Communication and Metacognition in Large Language Models

  Mark Steyvers, Catarina Belem, Padhraic Smyth · 23 June 2026

  Background. Large language models are increasingly used in settings where confident but incorrect answers can mislead users. Reliable uncertainty communication requires a form of metacognition: monitoring when one's own answers are likely to be correct. Yet models' stated confidence is often poorly …

• Evaluating Large Language Models for Hausa and Fongbe Machine Translation: Benchmarks, Failures, and Metric Reliability

  Mahounan Pericles Adjovi, Roald Eiselen, Prasenjit Mitra · 23 June 2026

  We investigate the translation quality of current large language models (LLMs) for English-to-Hausa and English-to-Fongbe - two typologically distinct West African languages from the Afroasiatic and Niger-Congo families respectively - and evaluate whether standard automatic metrics reliably reflect …

• EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning

  Jiawei Liu, Qisi Chen, Jianshu Zhang, Quan Liu, Defu Lian · 23 June 2026

  Large Language Models (LLMs) excel at complex reasoning through search algorithms, yet current strategies often suffer from massive token consumption due to redundant exploration of semantically equivalent steps. Existing semantic similarity methods struggle to accurately identify such equivalence i…

• Comparing Transformers and Hybrid Models at the Token Level

  Yanhong Li, William Merrill · 23 June 2026

  Hybrid language models that mix attention and recurrent layers have shown promise: theoretically, recurrent layers ameliorate the limitations of pure transformers on state tracking, and empirically, hybrids can outperform pure transformers in loss and downstream evaluations \citep{waleffe2024empiric…

• CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation

  Yuu Jinnai · 23 June 2026

  Nowadays, large multilingual translation models demonstrate impressive translation capabilities in the machine translation benchmarks. This raises a practical question to the developers: is it worth developing translation models specialized for a particular language pair if you only need to support …

• Which Pairs to Compare for LLM Post-Training?

  Jiangze Han, Vineet Goyal, Will Ma · 19 June 2026

  Preference-based post-training has become a central paradigm for aligning language models. A common data-collection strategy is to generate a small set of completions for each prompt and label the resulting comparison pairs. However, human preference labels are often much more expensive than generat…

• Toten: Knowledge-Based Ontological Tokenization Of Physical Quantities And Technical Notation In Brazilian Portuguese

  Antonio de Sousa Leit\~ao Filho; Allan Kardec Duailibe Barros Filho; Fabr\'icio Saul Lima; Selby Mykael Lima dos Santos; Rejani Bandeira Vieira Sousa · 19 June 2026

  Byte-Pair Encoding tokenization is statistically efficient for vocabulary compression, but semantically blind to structured technical entities, fragmenting physical quantities, numbers, units, and symbolic expressions into lexically arbitrary subwords. We present TOTEN, a knowledge-based ontological…

• AutoPass: Evidence-Guided LLM Agents for Compiler Performance Tuning

  Zepeng Li, Jie Ren, Zhanyong Tang, Jie Zheng, Zheng Wang · 19 June 2026

  Large Language Models (LLMs) show promise for code compilation tasks, but applying them to runtime performance tuning is difficult due to complex microarchitectural effects and noisy runtime measurements. We present AutoPass, a multi-agent framework for compiler performance tuning that uses compiler…

• Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees

  Herbert Woisetschl\"ager, Arastun Mammadli, Ryan Zhang, Shiqiang Wang · 19 June 2026

  Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost. Users expect high-quality responses, and in commercial settings this is formally codified in Service Level Agreements (SLAs), creating a fundamental tension betwe…

• Rethinking Cross-lingual Gaps from a Statistical Viewpoint

  Vihari Piratla, Purvam Jain, Darshan Singh, Trevor Cohn, Preethi Jyothi, Partha Talukdar · 18 June 2026

  Any piece of knowledge is usually expressed in one or a handful of natural languages on the web or in any large corpus. Large Language Models (LLMs) act as a bridge by acquiring knowledge from a source language and making it accessible when queried using target languages. A cross-lingual gap is a dr…

• When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

  Fred Philippy, Siwen Guo, Jacques Klein, Tegawend\'e F. Bissyand\'e · 18 June 2026

  Cross-lingual transfer in multilingual NLP has been widely explored in supervised fine-tuning contexts, where factors like data availability and linguistic similarity largely determine transfer quality. As the field shifts toward few-shot In-Context Learning (ICL), it is often presumed that insights…

• Approximate Structured Diffusion for Sequence Labelling

  Nicolas Floquet, Joseph Le Roux, Nadi Tomeh · 18 June 2026

  Sequence labelling, a core task of Natural Language Processing (NLP), consists in assigning each token of an input sentence a label. From a Machine Learning point of view, sequence labelling is often cast as a Linear-Chain Conditional Random Field (CRF) parametrised by a neural network. While th…

• DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQL

  Esteban Schafir, Xu Zheng, Hojat Allah Salehi, Zhuomin Chen, Mo Sha, Wei Cheng, Dongsheng Luo · 17 June 2026

  Large Language Models (LLMs) have demonstrated remarkable capabilities in translating natural language to SQL, yet existing methods still falter on complex queries requiring multi-step, data-aware reasoning. We introduce DecoSearch, a training-free framework that addresses this by routing each query…

• Translating the Untranslatable: An Operationalizable Ontology for Untranslatability

  Jacob Bremerman, Brihi Joshi, Hirona Arai, Xiang Ren, Jonathan May · 17 June 2026

  Untranslatability, cases where meaning cannot be directly preserved across languages, is well-studied in linguistics but underexplored in NLP. As machine translation (MT) systems improve on standard benchmarks, their limitations increasingly concentrate in such cases, where translation cannot be red…

• Models Take Notes at Prefill: KV Cache Can Be Editable and Composable

  Bojie Li · 17 June 2026

  Prefix caching reuses prefill only across an exactly shared prefix, so one changed field invalidates the entire downstream cache. Yet overwriting the field's own key/value vectors and reusing the rest leaves the model acting on the old value. The reason, established causally across four model famili…

• When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

  Fred Philippy, Siwen Guo, Jacques Klein, Tegawend\'e F. Bissyand\'e · 17 June 2026

  Cross-lingual transfer in multilingual NLP has been widely explored in supervised fine-tuning contexts, where factors like data availability and linguistic similarity largely determine transfer quality. As the field shifts toward few-shot In-Context Learning (ICL), it is often presumed that insights…

• Mordal: Automated Pretrained Model Selection for Vision Language Models

  Shiqi He, Insu Jang, Mosharaf Chowdhury · 17 June 2026

  Incorporating multiple modalities into large language models (LLMs) is a powerful way to enhance their understanding of non-textual data, enabling them to perform multimodal tasks. Vision language models (VLMs) form the fastest growing category of multimodal models because of their many practical us…

• MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task

  Jorge Iranzo-S\'anchez, Gerard Mas-Moll\`a, Adri\`a Gim\'enez, Jorge Civera, Albert Sanchis, Alfons Juan · 17 June 2026

  This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2026 Simultaneous Speech Translation track. Our submission utilizes the recently released Parakeet and Qwen 3.5 models to create a robust, cascaded solution for long-form SimulST through the use of…

• Enhancing LLM Safety Through a Theoretical Minimax Game Lens

  Yihe Deng, Yu Yang, Junkai Zhang, Wei Wang, Bo Li · 16 June 2026

  The rapid advancement of large language models (LLMs) necessitates effective mechanisms to ensure their responsible deployment by accurately distinguishing unsafe content from benign content. While substantial safety datasets are available in English, multilingual safety modeling remains underexplor…

• Koshur Diacritizer: A Byte-Level Sequence-to-Sequence Model for Kashmiri Diacritic Restoration

  Haq Nawaz Malik, Nahfid Nissar, Faizan Iqbal · 16 June 2026

  Kashmiri, an Indo-Aryan language written in a modified Perso-Arabic script, frequently omits diacritic marks in digital text, creating ambiguity and challenging downstream NLP applications. We present Koshur Diacritizer, a ByT5-small byte-level sequence-to-sequence model for restoring diacritics in …

• AC-ODM: Actor--Critic Online Data Mixing for Sample-Efficient LLM Pretraining

  Jing Ma, Chenhao Dang, Mingjie Liao · 16 June 2026

  Optimizing pretraining data composition is pivotal for LLM generalization. While dynamic mixing outperforms static strategies by capturing evolving training dynamics, current methods fail to reconcile computational efficiency with sample efficiency and structural flexibility for diverse pipelines.We…

• PolyKV: Heterogeneous Retention and Allocation for KV Cache Compression

  Chao Fei, Panos Kalnis · 16 June 2026

  KV cache compression is essential for reducing the memory cost of long-context large language model inference. Existing approaches, however, typically apply a single compression policy and a uniform cache budget across all transformer layers. This uniform design ignores the fact that different layer…