Natural Language Processing Techniques

Últimos artículos

• MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference

  Simla Burcu Harma, Danila Mishin, Zhengyuan Su, Ayan Chakraborty, Elizaveta Kostenok, Dongho Ha, Babak Falsafi, Martin Jaggi, Yunho Oh, Amir Yazdanbakhsh · 21 de julio de 2026

  4-bit quantization enables efficient LLM inference, but suffers from significant accuracy degradation due to outliers. Prior work addresses this problem via data rotation or mixed-precision integer quantization, but often relies on software-managed scaling and frequent dequantization, incurring subs…

• Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents

  Yitao Wu, Si Shen, Rui Yang, Hong Peng, Bin Hu · 21 de julio de 2026

  Verify-repair loops are a standard means for large language model (LLM) agents to correct faulty plans in code generation, mathematical reasoning, and tool use. When both the verifier and the repairer are noisy, repair can damage already-correct plans, and reported acceptance keeps rising while true…

• PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

  Hang Zhang, Warren J. Gross · 21 de julio de 2026

  Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reas…

• SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation

  Soumia Bouyahiaoui, Manel Kara laouar, Aicha Boutorh, Mohamed Hadj Ameur · 21 de julio de 2026

  Large Language Models (LLMs) generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major bottleneck. Recent compression methods mitigate this cost via token merging; however, these approaches often rely on indiscrimina…

• Feature Generation Using LLMs: An Evolutionary Algorithm Approach

  Aria Nourbakhsh, Beno\^it Alcaraz, Christoph Schommer · 21 de julio de 2026

  A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML a…

• Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory

  Qingcan Kang, Mingyang Liu, Shixiong Kai, Kaichao Liang, Zhentao Tang, Yuqi Cui, Tao Zhong, Mingxuan Yuan · 21 de julio de 2026

  Language agents depend on memory across interactions. However, the limited context windows of large language models (LLMs) and their inference costs constrain how much memory can be used at once. Existing systems mainly follow two strategies: memory retention and memory consolidation. Retention keep…

• Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment

  Daniel Cersosimo · 21 de julio de 2026

  AI democratization is not primarily a question of matching frontier-scale generality; it is a question of whether capable models can be selected, audited, and specialized under hardware and governance constraints that ordinary institutions can actually satisfy. This paper studies that problem throug…

• Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances

  Hanlei Zhang, Hua Xu, Fei Long, Xin Wang, Kai Gao · 21 de julio de 2026

  Discovering the semantics of multimodal utterances is essential for understanding human language and enhancing human-machine interactions. Existing methods manifest limitations in leveraging nonverbal information for discerning complex semantics in unsupervised scenarios. This paper introduces a nov…

• FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

  Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao · 21 de julio de 2026

  Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial. Prior work has attempted to unlock inter-block parallelism through post-training methods, but achieves only mode…

• PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

  Yuchen Yang, Yifan Zhao, Anisha Dasgupta, Sasa Misailovic · 20 de julio de 2026

  Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exhibit a tension between the GPU memory requirements of the model weights and the growing KV cache. We propose PagedWeight…

• Rethinking the Global Knowledge of CLIP in Training-Free Open-Vocabulary Semantic Segmentation

  Jingyun Wang, Cilin Yan, Guoliang Kang · 20 de julio de 2026

  Recent works modify CLIP to perform open-vocabulary semantic segmentation in a training-free manner (TF-OVSS). In vanilla CLIP, patch-wise image representations mainly encode homogeneous image-level properties, which hinders the application of CLIP to the dense prediction task. Previous TF-OVSS work…

• VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

  Shahrzad Esmat, Dhawal Shah, Ali Jannesari · 20 de julio de 2026

  The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference. Two leading training-free families are both structurally limited: token-selection methods (SnapKV, Ada-KV) score importance from an observation window and evict low-scoring tokens, but evicti…

• Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization

  Zhiqi Gao, Albert Ge, Alexander Berenbeim, Nathaniel D. Bastian, Frederic Sala · 17 de julio de 2026

  Text-to-optimization requires two separable capabilities: modeling -- choosing the right optimization structure -- and binding -- grounding every coefficient, index, and parameter in the concrete problem data. We study this via Text2Opt-Bench, a scalable benchmark of solver-verified optimization pro…

• In-Place Tokenizer Expansion for Pre-trained LLMs

  Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera, Simon S. Lee, Paul Pak, Aditya Tadimeti, Tim Seyde, Maxime Labonne, Alexander Amini, Mathias Lechner · 17 de julio de 2026

  A tokenizer fixed at the start of pre-training allocates vocabulary in proportion to the pre-training corpus, reflecting the deployment priorities at that time. When those priorities shift, languages added later are split into many more tokens per word, which can raise latency, compute, and energy c…

• Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

  Xuhui Zhan, Tyler Derr · 16 de julio de 2026

  Traditional multimodal learning approaches rely on alignment pre-training to bridge vision and language modalities, typically by projecting visual features into discrete text token spaces using large-scale image--text data. We revisit this design choice and propose Inverse-LLaVA, a multimodal archit…

• Infinity-Parser2 Technical Report

  Zuming Huang, Jun Huang, Kexuan Ren, Baode Wang, Weizhen Li, Jianming Feng, Yu Wang, Yichen Yao, Shijun Lin, Yige Tang, Cheng Peng, Weidi Xu, Wei Chu, Yinghui Xu, Yuan Qi · 16 de julio de 2026

  We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora. Our contributions are threefold. First, we bui…

• DarwinLM: Evolutionary Structured Pruning of Large Language Models

  Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Zhiqiang Shen, Dan Alistarh · 16 de julio de 2026

  Large Language Models (LLMs) have achieved significant success across various NLP tasks. However, their massive computational costs limit their widespread use, particularly in real-time applications. Structured pruning offers an effective solution by compressing models and directly providing end-to-…

• Growing a Tail: Increasing Output Diversity in Large Language Models

  Michal Shur-Ofry, Bar Horowitz-Amsalem, Adir Rahamim, Yonatan Belinkov · 15 de julio de 2026

  How diverse are the outputs of large language models when diversity is desired? We examine the diversity of responses of several language models to questions with multiple possible answers, comparing them with human responses. Our findings suggest that models' responses are highly concentrated, refl…

• A Shared Subcircuit Lets LLMs Count Down Across Tasks

  Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen · 15 de julio de 2026

  Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires tracking how many tokens remain before a target. In this work, we identify in Llama-3.1-70B-Instruct a general mechanism for …

• Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

  Huan Zhu · 14 de julio de 2026

  Self-refinement often fails to strengthen few-shot inductive reasoning in large language models. Prompting a model to explicitly state its inferred rule does little on its own. What actually matters is a structurally enforced isolation between reasoning stages, so that information can only pass betw…

• LeRoPE: Learnable RoPE Frequencies Improve Language Modeling

  Petros Karypis, Sean O'Brien, Shreyas Kadekodi, Rui Zhu, Julian McAuley · 14 de julio de 2026

  Rotary Positional Encodings (RoPE) are currently the most popular positional encodings used in modern language models. RoPE rotates two-dimensional chunks of query and key vectors, operating as a function of their relative positional offset. The position-wise rates of rotation in RoPE typically foll…

• Agentic Routing: The Harness-Native Data Flywheel

  Xinchen Liu, Hang Zhou, Yingjie Zong, Yuchuan Tian, Liuyang Song, Shuo Zhang, Yulong Li, Wei He, Mengyu Zheng, Runke Liu, Siyang Cheng, Xiang Kuang, Hailin Hu, Kai Han, Yunhe Wang · 14 de julio de 2026

  Large language model agents are increasingly executed not by a single model call, but by an execution harness that manages observation, context, control, action, state, and verification. At the same time, frontier and open models are becoming structurally specialized: a model that is strong at code …

• HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA

  Xinyue Chen, Pengyu Gao, Jiangjiang Song, Xiaoyang Tan · 14 de julio de 2026

  Retrieval-augmented generation (RAG) has rapidly advanced the language model field, particularly in question-answering (QA) systems. By integrating external documents during the response generation phase, RAG significantly enhances the accuracy and reliability of language models. This method elevate…

• Reliability Scaling Laws for Quantized Large Language Models

  Sirine Ayadi, S\'andor Dar\'oczi, Stephan G\"unnemann, Bertrand Charpentier · 14 de julio de 2026

  Quantization is a powerful strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters. While quantized LLMs achieve state-of-the-art performance on unperturbed inputs using standard predictive metrics, their performance on perturbed input…

• IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMs

  Jinwei He, Feng Lu · 13 de julio de 2026

  Large language models (LLMs) have developed rapidly, and their reasoning capabilities have become a hot research topic. However, there is still limited exploration of abductive reasoning. The multi-perspective and multi-level of causes is one of the core challenges of abductive reasoning, which cann…