Natural Language Processing Techniques

Latest papers

• From Noise to Signal to Selbstzweck: Reframing Human Label Variation in the Era of Post-training in NLP

  Shanshan Xu, Santosh T. Y. S. S, Barbara Plank · 19 January 2026

  Human Label Variation (HLV) refers to legitimate disagreement in annotation that reflects the diversity of human perspectives rather than mere error. Long treated in NLP as noise to be eliminated, HLV has only recently been reframed as a signal for improving model robustness. With the rise of large …

• MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extraction

  Wei-Chieh Huang, Cornelia Caragea · 19 January 2026

  Implicit Attribute Value Extraction (AVE) is essential for accurately representing products in e-commerce, as it infers latent attributes from multimodal data. Despite advances in multimodal large language models (MLLMs), implicit AVE remains challenging due to the complexity of multidimensional dat…

• Better Language Models Exhibit Higher Visual Alignment

  Jona Ruthardt, Gertjan J. Burghouts, Serge Belongie, Yuki M. Asano · 19 January 2026

  How well do text-only large language models (LLMs) align with the visual world? We present a systematic evaluation of this question by incorporating frozen representations of various language models into a discriminative vision-language framework and measuring zero-shot generalization to novel conce…

• HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning

  Ziang Cui, Mengran Yu, Tianjiao Li, Chenyu Shi, Yingxuan Shi, Lusheng Zhang, Hongwei Lin · 16 January 2026

  Large Language Models (LLMs) have achieved remarkable strides in multilingual translation but are hindered by a systemic cross-lingual verbosity bias, rendering them unsuitable for strict time-constrained tasks like subtitling and dubbing. Current prompt-engineering approaches struggle to resolve th…

• VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Models

  Aman Gupta, Denny O'Shea, Fazl Barez · 16 January 2026

  Large language models (LLMs) are increasingly being used for tasks where outputs shape human decisions, so it is critical to verify that their responses consistently reflect desired human values. Humans, as individuals or groups, don't agree on a universal set of values, which makes evaluating value…

• Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG

  David Samuel Setiawan, Rapha\"el Merx, Jey Han Lau · 16 January 2026

  Neural Machine Translation (NMT) models for low-resource languages suffer significant performance degradation under domain shift. We quantify this challenge using Dhao, an indigenous language of Eastern Indonesia with no digital footprint beyond the New Testament (NT). When applied to the unseen Old…

• Small Open Models Achieve Near Parity with Large Models in Low Resource Literary Translation at a Fraction of the Cost

  Mihai Nadas, Laura Diosan, Andreea Tomescu, Andrei Piscoran · 16 January 2026

  Literary translation has recently gained attention as a distinct and complex task in machine translation research. However, the translation by small open models remains an open problem. We contribute to this ongoing research by introducing TinyFabulist Translation Framework (TF2), a unified framewor…

• Beyond Uniform SVD:Dual-Level Optimization across Columns and Modules for LLM Compression

  Lin Xv, Xian Gao, Ting Li, Yuzhuo Fu · 15 January 2026

  Low-rank decomposition, particularly Singular Value Decomposition (SVD), is a pivotal technique for mitigating the storage and computational demands of Large Language Models (LLMs). However, prevalent SVD-based approaches overlook the critical phenomenon that decomposition errors exhibit significant…

• TranslateGemma Technical Report

  Mara Finkelstein, Isaac Caswell, Tobias Domhan, Jan-Thorsten Peter, Juraj Juraska, Parker Riley, Daniel Deutsch, Cole Dilanni, Colin Cherry, Eleftheria Briakou, Elizabeth Nielsen, Jiaming Luo, Kat Black, Ryan Mullins, Sweta Agrawal, Wenda Xu, Erin Kats, Stephane Jaskiewicz, Markus Freitag, David Vilar · 15 January 2026

  We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the translation task, we employ a two-stage fine-tuning process. First, supervised fine-tuning is performed using a rich mixt…

• Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats

  Manyi Zhang, Ji-Fu Li, Zhongao Sun, Haoli Bai, Hui-Ling Zhen, Zhenhua Dong, Xianzhi Yu · 15 January 2026

  Microscaling Floating-Point (MXFP) has emerged as a promising low-precision format for large language models (LLMs). Despite various post-training quantization (PTQ) algorithms being proposed, they mostly focus on integer quantization, while their applicability and behavior under MXFP formats remain…

• Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs

  Farnaz Kohankhaki, D. B. Emerson, Jacob-Junqi Tian, Laleh Seyyed-Kalantari, Faiza Khan Khattak · 15 January 2026

  Bias in large language models (LLMs) has many forms, from overt discrimination to implicit stereotypes. Counterfactual bias evaluation is a widely used approach to quantifying bias and often relies on template-based probes that explicitly state group membership. It aims to measure whether the outcom…

• DiffSampling: Enhancing Diversity and Accuracy in Neural Text Generation

  Giorgio Franceschelli, Mirco Musolesi · 15 January 2026

  Despite their growing capabilities, language models still frequently reproduce content from their training data, generate repetitive text, and favor common grammatical patterns and vocabulary. A possible cause is the decoding strategy: the most common strategies either consider only the most probabl…

• Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality

  Zhenglun Kong, Yize Li, Fanhu Zeng, Lei Xin, Shvat Messica, Xue Lin, Pu Zhao, Manolis Kellis, Hao Tang, Marinka Zitnik · 14 January 2026

  In Transformer architectures, tokens\textemdash discrete units derived from raw data\textemdash are formed by segmenting inputs into fixed-length chunks. Each token is then mapped to an embedding, enabling parallel attention computations while preserving the input's essential information. Due to the…

• Vocabulary Expansion of Large Language Models via Kullback-Leibler-Based Self-Distillation

  Max Rehman Linder · 14 January 2026

  Large pre-trained language models often struggle to incorporate new domain-specific terminology when fine-tuned on small, specialized corpora. In this work, we address the challenge of vocabulary expansion in frozen LLMs by introducing a mathematically grounded method for knowledge distillation via …

• Semantic Gravity Wells: Why Negative Constraints Backfire

  Shailesh Rana · 14 January 2026

  Negative constraints (instructions of the form "do not use word X") represent a fundamental test of instruction-following capability in large language models. Despite their apparent simplicity, these constraints fail with striking regularity, and the conditions governing failure have remained poorly…

• Hierarchical Sparse Plus Low Rank Compression of LLM

  Pawan Kumar, Aditi Gupta · 14 January 2026

  Modern large language models (LLMs) place extraordinary pressure on memory and compute budgets, making principled compression indispensable for both deployment and continued training. We present Hierarchical Sparse Plus Low-Rank (HSS) compression, a two-stage scheme that (i) removes the largest-magn…

• Foundations of LLM Knowledge Materialization: Termination, Reproducibility, Robustness

  Luca Giordano, Simon Razniewski · 14 January 2026

  Large Language Models (LLMs) encode substantial factual knowledge, yet measuring and systematizing this knowledge remains challenging. Converting it into structured format, for example through recursive extraction approaches such as the GPTKB methodology (Hu et al., 2025b), is still underexplored. K…

• Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference

  Rei Taniguchi, Yuyang Dong, Makoto Onizuka, Chuan Xiao · 13 January 2026

  Due to the prevalence of large language models (LLMs), key-value (KV) cache reduction for LLM inference has received remarkable attention. Among numerous works that have been proposed in recent years, layer-wise token pruning approaches, which select a subset of tokens at particular layers to retain…

• Cost-Awareness in Tree-Search LLM Planning: A Systematic Study

  Zihao Zhang, Hui Wei, Kenan Jiang, Shijia Pan, Shu Kai, Fei Liu · 13 January 2026

  Planning under resource constraints is central to real-world decision making, yet most large language model (LLM) planners assume uniform action costs. We systematically analyze whether tree-search LLM planners are cost-aware and whether they efficiently generate budget-feasible plans. In contrast t…

• Large Language Model-Based Automatic Formulation for Stochastic Optimization Models

  Amirreza Talebi · 13 January 2026

  This paper presents an integrated systematic study of the performance of large language models (LLMs), specifically ChatGPT, for automatically formulating and solving Stochastic Optimization (SO) problems from natural language descriptions. Focusing on three key categories, individual chance-constra…

• Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation

  Baban Gain, Dibyanayan Bandyopadhyay, Asif Ekbal, Trilok Nath Singh · 13 January 2026

  Large Language Models (LLMs) are rapidly reshaping machine translation (MT), particularly by introducing instruction-following, in-context learning, and preference-based alignment into what has traditionally been a supervised encoder-decoder paradigm. This survey provides a comprehensive and up-to-d…

• Flexible Realignment of Language Models

  Wenhong Zhu, Ruobing Xie, Weinan Zhang, Rui Wang · 13 January 2026

  Realignment becomes necessary when a language model (LM) fails to meet expected performance. We propose a flexible realignment framework that supports quantitative control of alignment degree during training and inference. This framework incorporates Training-time Realignment (TrRa), which efficient…

• Restoring Rhythm: Punctuation Restoration Using Transformer Models for Bangla, A Low-Resource Language

  Md Obyedullahil Mamun, Md Adyelullahil Mamun, Arif Ahmad, Md. Imran Hossain Emu · 13 January 2026

  Punctuation restoration enhances the readability of text and is critical for post-processing tasks in Automatic Speech Recognition (ASR), especially for low-resource languages like Bangla. In this study, we explore the application of transformer-based models, specifically XLM-RoBERTa-large, to autom…

• LLMs Enable Bag-of-Texts Representations for Short-Text Clustering

  I-Fan Lin, Faegheh Hasibi, Suzan Verberne · 13 January 2026

  In this paper, we propose a training-free method for unsupervised short text clustering that relies less on careful selection of embedders than other methods. In customer-facing chatbots, companies are dealing with large amounts of user utterances that need to be clustered according to their intent.…

• NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learning

  Zhongtao Miao, Kaiyan Zhao, Masaaki Nagata, Yoshimasa Tsuruoka · 13 January 2026

  Neologism-aware machine translation aims to translate source sentences containing neologisms into target languages. This field remains underexplored compared with general machine translation (MT). In this paper, we propose an agentic framework, NeoAMT, for neologism-aware machine translation using a…