Natural Language Processing Techniques

Neueste Paper

• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification

  Sercan Karaka\c{s}, Yusuf \c{S}im\c{s}ek · 8. Juni 2026

  Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing because they often share the same surface form as fully literal verb-object combinations while functioning as a single, partially idiomatic predicate. We frame Turkish LVC detection as a binary clas…

• Scalable Joint Resource Allocation for SLO-Constrained LLM Inference in Heterogeneous GPU Clouds

  Jiaming Cheng, Duong Tung Nguyen · 8. Juni 2026

  Serving large language model (LLM) inference in cloud environments requires jointly optimizing model selection, GPU provisioning, parallelism configuration, and workload routing under latency, accuracy, memory, and budget constraints. While mixed-integer linear programming (MILP) can model this prob…

• AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

  Beshr IslamBouli, David Jin · 8. Juni 2026

  Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference. Existing PTQ methods, such as AWQ and GPTQ, improve how weights are mapped onto a fixed 4-bit grid through scaling, clipping, or error compensation. To further im…

• ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning

  Vladislav Smirnov (MBZUAI), Chieu Nguyen (MBZUAI), Sergey Senichev (Independent Researcher), Minh Ngoc Ta (MBZUAI), Ekaterina Fadeeva (ETH Z\"urich), Artem Vazhentsev (MBZUAI), Daria Galimzianova (MBZUAI), Nikolai Rozanov (MBZUAI, Imperial College London), Viktor Mazanov (Innopolis University), Jingwei Ni (ETH Z\"urich), Tianyi Wu (NUS), Igor Kiselev (Accenture), Mrinmaya Sachan (ETH Z\"urich), Iryna Gurevych (MBZUAI), Preslav Nakov (MBZUAI), Timothy Baldwin (MBZUAI), Artem Shelmanov (MBZUAI) · 8. Juni 2026

  Test-time compute (TTC) scaling has emerged as a powerful paradigm for improving large language model (LLM) reasoning by allocating additional compute during inference, e.g., via multi-sample generation and verifier-based reranking. Existing TTC scaling strategies and reasoning scorers remain fragme…

• Trace Reconstruction with Language Models

  Franziska Weindel, Michael Girsch, Reinhard Heckel · 8. Juni 2026

  The general trace reconstruction problem seeks to recover an original sequence from its noisy copies independently corrupted by insertions, deletions, and substitutions. This problem arises in applications such as DNA data storage, a promising storage medium due to its high information density and l…

• Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards

  Shihao Zhang, Xiaoman Wang, Yuan Liu, Yunshi Lan, Weining Qian · 8. Juni 2026

  Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to captur…

• DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention

  Najmul Hasan, Prashanth BusiReddyGari · 5. Juni 2026

  We present DPBench, a benchmark for evaluating coordination in multi-agent systems built from large language models. Existing benchmarks measure task-level success under a fixed protocol; the structural conditions under which coordination succeeds or fails at all have not been characterised. DPBench…

• Better Literary Translation: A Multi-Aspect Data Generation and LLM Training Approach

  Zhihao Lin, Ziqi Zhu, Hao Huang, Guanghui Wang, Peiyang He · 5. Juni 2026

  Literary translation poses unique challenges due to the scarcity of high-quality annotated data and the need to balance expression fluency with literary effect. We present a multi-aspect iterative refinement framework that generates high-quality translation references and preference data through spe…

• QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

  Jianxin Yan, Wangze Ni, Zhenxin Li, Jiabao Jin, Zhitao Shen, Haoyang Li, Jia Zhu, Peng Cheng, Xuemin Lin, Lei Chen, Kui Ren · 5. Juni 2026

  Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG cache fusion reduces this cost by reusing precomputed key-value (KV) caches for…

• LLM Compression with Jointly Optimizing Architectural and Quantization choices

  Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi, Phuong Hoai Ha · 4. Juni 2026

  Deploying large language models (LLMs) is challenging due to their significant memory and computational requirements. While some methods address this by developing small or tiny language models from scratch, these approaches demand extensive GPU training. Compressing pre-trained LLMs for edge device…

• Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them)

  Nizar Islah, Istabrak Abbes, Irina Rish, Sarath Chandar, Eilif B. Muller · 4. Juni 2026

  When post-trained language models fail on reasoning problems, the common test-time-scaling response is to spend more compute on additional attempts, and the failed traces play no further role. We argue this discards a crucial signal; some failures come from unlucky sampling, where more rollouts help…

• ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents

  Zhigen Li, Jianxiang Peng, Yanmeng Wang, Yong Cao, Tianhao Shen, Minghui Zhang, Linxi Su, Shang Wu, Yihang Wu, Yuqian Wang, Ye Wang, Wei Hu, Jianfeng Li, Shaojun Wang, Jing Xiao, Deyi Xiong · 4. Juni 2026

  Dialogue agents powered by Large Language Models (LLMs) show superior performance in various tasks. Despite the better user understanding and human-like responses, their **lack of controllability** remains a key challenge, often leading to unfocused conversations or task failure. To address this, we…

• Token Rankings are Unforgeable Language Model Signatures

  Matthew Finlayson, Andreas Grivas, Xiang Ren, Swabha Swayamdipta · 4. Juni 2026

  Language model parameters are known to impose unique (to each model) geometric constraints on their logit outputs, which serves as a signature that identifies the model, but also leaks the model's final layer parameters when an API distributes logits. We investigate more restrictive APIs that expose…

• Test-time reward-guided alignment of language models by importance sampling on pre-logit space

  Sekitoshi Kanai, Tsukasa Yoshida, Hiroshi Takahashi, Haru Kuroki, Kazumune Hashimoto · 4. Juni 2026

  Test-time alignment of large language models (LLMs) attracts attention because fine-tuning of LLMs requires high computational costs. In this paper, we propose a new test-time reward-guided alignment method called adaptive importance sampling on pre-logits (AISP) on the basis of the sampling-based m…

• Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution

  Qiao Xiao, Alan Ansell, Boqian Wu, Lu Yin, Mykola Pechenizkiy, Shiwei Liu, Decebal Constantin Mocanu · 3. Juni 2026

  Sparse large language models (LLMs) offer an attractive direction toward efficient deployment, but adapting them to downstream tasks remains challenging. The central difficulty is to enable effective task adaptation without sacrificing the efficiency advantages of sparsity. Existing fine-tuning meth…

• AlignAtt4LLM: Fast AlignAtt for Decoder-Only LLMs at IWSLT 2026 Simultaneous Speech Translation Task

  Quentin Fuxa, Dominik Mach\'a\v{c}ek · 3. Juni 2026

  We describe AlignAtt4LLM, an IWSLT 2026 simultaneous speech translation system for English to German, Italian, and Chinese. The system is a synchronous cascade: Qwen3-ASR with forced alignment produces an incrementally updated source transcript, and Gemma-4 E4B-it translates that prefix under an MT-…

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language

  Muhammad Ali · 3. Juni 2026

  We present BaltiVoice, a 16.8-hour read-speech corpus for Balti (ISO 639-3: bft), a Tibetic language spoken in Gilgit-Baltistan, Pakistan, with no prior publicly available ASR resources. The corpus contains 10,060 validated utterances in native Nastaliq script, derived from Mozilla Common Voice reco…

• Calibration Data Trade-offs Across Capability Dimensions: Why Multi-Source Mixing Matters for High-Sparsity LLM Pruning

  Hu Xu, Zhaolong Xing, Congcong Liu, Jiaxing Wang, Zhida Jiang, Junshi Huang, Zhen Chen, Jianfeng Xu · 3. Juni 2026

  Post-training pruning compresses large language models to high sparsity using a small unlabelled calibration set, and recent work has concluded that the choice of calibration source has only modest impact on averaged post-pruning accuracy. We ask whether this conclusion survives once calibration imp…

• Alignment-Aware Decoding

  Fr\'ed\'eric Berdoz, Luca A. Lanzend\"orfer, Ren\'e Caky, Roger Wattenhofer · 3. Juni 2026

  Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving alignment, typically through training-time or prompt-based interventions. In this paper, we introduce alignment-aware dec…

• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference

  Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li · 3. Juni 2026

  Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely committed together. Fast-dLLM addressed this with KV caching and confidence-guided parallel decoding, but its decoding theory uses a homogeneous high-co…

• Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing

  Mehmet Utku Colak · 3. Juni 2026

  AI-assisted coding agents are bottlenecked by input-token cost. Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts. Existing approaches act reactively by compressing already-bloated contexts …

• Sample-Size Scaling of the African Languages NLI Evaluation

  Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion, Jesujuwon Egbewale, Hannah Nwokocha · 3. Juni 2026

  African languages have very little labelled data, and it is unclear if augmenting the quantity of annotation data reliably enhances downstream performance. The study is a systematic sample-size scaling study of natural language inference (NLI) on 16 African languages based on the AfriXNLI benchmark.…

• The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

  Huixue Zhou, Hengrui Gu, Xi Liu, Kaixiong Zhou, Mingfu Liang, Yongkang Xiao, Srinivas Govindan, Piyush Chawla, Jiyan Yang, Xiangfei Meng, Huayu Li, Buyun Zhang, Liang Luo, Wen-Yen Chen, Yiping Han, Bo Long, Rui Zhang, Tianlong Chen · 3. Juni 2026

  The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) wi…

• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference

  Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li · 3. Juni 2026

  Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely committed together. Fast-dLLM addressed this with KV caching and confidence-guided parallel decoding, but its decoding theory uses a homogeneous high-co…

• KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks

  Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang, Lukas Cavigelli · 3. Juni 2026

  Test-time scaling is a powerful approach to obtain better reasoning in large language models, but it becomes memory-bottlenecked during long-horizon decoding, as the KV-cache grows. KV-cache quantization can help improve this, but current methods are evaluated under prefill-like settings and errors …