Natural Language Processing Techniques

Derniers papiers

• Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents

  Yushu Li, Wenlong Deng, Jiajin Li, Xiaoxiao Li · 16 mars 2026

  Test-time scaling has become a dominant paradigm for improving LLM agent reliability, yet current approaches treat compute as an abundant resource, allowing agents to exhaust token and tool budgets on redundant steps or dead-end trajectories. Existing budget-aware methods either require expensive fi…

• LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation

  Himel Ghosh, Nick Elias Werner · 16 mars 2026

  As large language models (LLMs) are deployed widely, detecting and understanding bias in their outputs is critical. We present LLM BiasScope, a web application for side-by-side comparison of LLM outputs with real-time bias analysis. The system supports multiple providers (Google Gemini, DeepSeek, Mi…

• Is Human Annotation Necessary? Iterative MBR Distillation for Error Span Detection in Machine Translation

  Boxuan Lyu, Haiyue Song, Zhi Qu · 16 mars 2026

  Error Span Detection (ESD) is a crucial subtask in Machine Translation (MT) evaluation, aiming to identify the location and severity of translation errors. While fine-tuning models on human-annotated data improves ESD performance, acquiring such data is expensive and prone to inconsistencies among a…

• Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language

  Remigiusz Kinas, Pawe{\l} Kiszczak, Sergio P. Perez, Krzysztof Ociepa, {\L}ukasz Flis, Krzysztof Wr\'obel, Adrian Gwo\'zdziej · 13 mars 2026

  This report details the creation of Bielik-Minitron-7B, a compressed 7.35B parameter version of the Bielik-11B-v3.0 model, specifically optimized for European languages. By leveraging a two-stage compression methodology inspired by the NVIDIA Minitron approach, we combined structured hybrid pruning …

• Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct

  Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong, Nan Jiang, Zheyuan Hu, Weijian Luo, Wei Deng, Guang Lin · 13 mars 2026

  Fast and high-quality language generation is the holy grail that people pursue in the age of AI. In this work, we introduce Discrete Diffusion Divergence Instruct (DiDi-Instruct), a training-based method that initializes from a pre-trained diffusion large language model (dLLM) and distills a few-ste…

• Speculative Decoding Scaling Laws (SDSL): Throughput Optimization Made Simple

  Amirhossein Bozorgkhoo, Igor Molybog · 13 mars 2026

  Speculative decoding is a technique that uses multiple language models to accelerate infer- ence. Previous works have used an experi- mental approach to optimize the throughput of the inference pipeline, which involves LLM training and can be costly. This study of spec- ulative decoding proposes a t…

• Markovian Generation Chains in Large Language Models

  Mingmeng Geng, Amr Mohamed, Guokan Shang, Michalis Vazirgiannis, Thierry Poibeau · 13 mars 2026

  The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous…

• IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse

  Yushi Bai, Qian Dong, Ting Jiang, Xin Lv, Zhengxiao Du, Aohan Zeng, Jie Tang, Juanzi Li · 13 mars 2026

  Long-context agentic workflows have emerged as a defining use case for large language models, making attention efficiency critical for both inference speed and serving cost. Sparse attention addresses this challenge effectively, and DeepSeek Sparse Attention (DSA) is a representative production-grad…

• ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps

  Sicheng Feng, Song Wang, Shuyi Ouyang, Lingdong Kong, Zikai Song, Jianke Zhu, Huan Wang, Xinchao Wang · 13 mars 2026

  Multimodal large language models (MLLMs) have demonstrated significant progress in semantic scene understanding and text-image alignment, with reasoning variants enhancing performance on more complex tasks involving mathematics and logic. To bridge this gap, we introduce ReasonMap, a novel benchmark…

• The Generation-Recognition Asymmetry: Six Dimensions of a Fundamental Divide in Formal Language Theory

  Romain Peyrichou · 12 mars 2026

  Every formal grammar defines a language and can in principle be used in three ways: to generate strings (production), to recognize them (parsing), or -- given only examples -- to infer the grammar itself (grammar induction). Generation and recognition are extensionally equivalent -- they characteriz…

• EvoSchema: Towards Text-to-SQL Robustness Against Schema Evolution

  Tianshu Zhang, Kun Qian, Siddhartha Sahai, Yuan Tian, Shaddy Garg, Huan Sun, Yunyao Li · 12 mars 2026

  Neural text-to-SQL models, which translate natural language questions (NLQs) into SQL queries given a database schema, have achieved remarkable performance. However, database schemas frequently evolve to meet new requirements. Such schema evolution often leads to performance degradation for models t…

• Automated evaluation of LLMs for effective machine translation of Mandarin Chinese to English

  Yue Zhang, Rodney Beard, John Hawkins, Rohitash Chandra · 12 mars 2026

  Although Large Language Models (LLMs) have exceptional performance in machine translation, only a limited systematic assessment of translation quality has been done. The challenge lies in automated frameworks, as human-expert-based evaluations can be time-consuming, given the fast-evolving LLMs and …

• Adaptive Engram Memory System for Indonesian Language Model: Generative AI Based on TOBA LM for Batak and Minang Language

  Hokky Situngkir, Kevin Siringoringo, Andhika Bernard Lumbantobing · 12 mars 2026

  This study presents TOBA-LM, a trilingual language model based on GPT-2 architecture with 1.2 billion parameters, trained on a corpus encompassing Indonesian, Batak, and Minangkabau using syllabic-agglutinative tokenization. The architecture integrates an Engram Memory mechanism, an adaptive n-gram-…

• ConLID: Supervised Contrastive Learning for Low-Resource Language Identification

  Negar Foroutan, Jakhongir Saydaliev, Ye Eun Kim, Antoine Bosselut · 11 mars 2026

  Language identification (LID) is a critical step in curating multilingual LLM pretraining corpora from web crawls. While many studies on LID model training focus on collecting diverse training data to improve performance, low-resource languages -- often limited to single-domain data, such as the Bib…

• UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models

  Xiaojie Gu, Ziying Huang, Jia-Chen Gu, Kai Zhang · 11 mars 2026

  Lifelong learning enables large language models (LLMs) to adapt to evolving information by continually updating their internal knowledge. An ideal system should support efficient, wide-ranging updates while preserving existing capabilities and ensuring reliable deployment. Model editing stands out a…

• OPENXRD: A Comprehensive Benchmark Framework for LLM/MLLM XRD Question Answering

  Ali Vosoughi, Ayoub Shahnazari, Yufeng Xi, Zeliang Zhang, Griffin Hess, Chenliang Xu, Niaz Abdolrahim · 11 mars 2026

  We introduce OPENXRD, a comprehensive benchmarking framework for evaluating large language models (LLMs) and multimodal LLMs (MLLMs) in crystallography question answering. The framework measures context assimilation, or how models use fixed, domain-specific supporting information during inference. T…

• Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios

  Namrata Patil Gurav, Akashdeep Ranu, Archchana Sindhujan, Diptesh Kanojia · 10 mars 2026

  Quality Estimation (QE) is essential for assessing machine translation quality in reference-less settings, particularly for domain-specific and low-resource language scenarios. In this paper, we investigate sentence-level QE for English to Indic machine translation across four domains (Healthcare, L…

• EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs

  Chang Han, Yijie Hu, Jingling Liu · 10 mars 2026

  Autoregressive decoding remains a primary bottleneck in large language model (LLM) serving, motivating speculative decoding methods that reduce expensive teacher-model invocations by verifying multiple candidate tokens per step. Tree-structured speculation further increases parallelism, but is often…

• LycheeCluster: Efficient Long-Context Inference with Structure-Aware Chunking and Hierarchical KV Indexing

  Dongfang Li, Zixuan Liu, Gang Lin, Baotian Hu, Min Zhang · 10 mars 2026

  The quadratic complexity of the attention mechanism and the substantial memory footprint of the Key-Value (KV) cache present severe computational and memory challenges for Large Language Models (LLMs) processing long contexts. Existing retrieval-based methods often compromise semantic integrity thro…

• HDLxGraph: Bridging Large Language Models and HDL Repositories via HDL Graph Databases

  Pingqing Zheng (Katie), Jiayin Qin (Katie), Fuqi Zhang (Katie), Niraj Chitla (Katie), Zishen Wan (Katie), Shang Wu (Katie), Yu Cao (Katie), Caiwen Ding (Katie), Yang (Katie), Zhao · 10 mars 2026

  Retrieval Augmented Generation (RAG) is an essential agent for Large Language Model (LLM) aided Description Language (HDL) tasks, addressing the challenges of limited training data and prohibitively long prompts. However, its performance in handling ambiguous queries and real-world, repository-level…

• Not all tokens are needed(NAT): token efficient reinforcement learning

  Hejian Sang, Yuanda Xu, Zhengze Zhou, Ran He, Zhipeng Wang · 10 mars 2026

  Reinforcement learning (RL) has become a key driver of progress in large language models, but scaling RL to long chain-of-thought (CoT) trajectories is increasingly constrained by backpropagation over every generated token. Even with optimized rollout engines, full-token updates can consume a large …

• GRADIEND: Feature Learning within Neural Networks Exemplified through Biases

  Jonathan Drechsel, Steffen Herbold · 10 mars 2026

  AI systems frequently exhibit and amplify social biases, leading to harmful consequences in critical areas. This study introduces a novel encoder-decoder approach that leverages model gradients to learn a feature neuron encoding societal bias information such as gender, race, and religion. We show t…

• Structural Inference: Interpreting Small Language Models with Susceptibilities

  Garrett Baker, George Wang, Jesse Hoogland, Daniel Murfet · 10 mars 2026

  We develop a linear response framework for interpretability that treats a neural network as a Bayesian statistical mechanical system. A small perturbation of the data distribution, for example shifting the Pile toward GitHub or legal text, induces a first-order change in the posterior expectation of…

• Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation

  Peter Pol\'ak, Sara Papi, Luisa Bentivogli, Ond\v{r}ej Bojar · 9 mars 2026

  Simultaneous speech-to-text translation systems must balance translation quality with latency. Although quality evaluation is well established, latency measurement remains a challenge. Existing metrics produce inconsistent results, especially in short-form settings with artificial presegmentation. W…

• Algorithmic Collusion by Large Language Models

  Sara Fish, Yannai A. Gonczarowski, Ran I. Shorrer · 9 mars 2026

  We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs). In oligopoly settings, LLM-based pricing agents quickly and autonomously reach supracompetitive prices and profits. Variation in seemingly innocuous phrases in LLM instructions ("prompts") substantially inf…