Natural Language Processing Techniques

Últimos artículos

• CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning

  Alexandra Dragomir, Florin Brad, Radu Tudor Ionescu · 12 de enero de 2026

  Large language models (LLMs) have demonstrated competitive performance in zero-shot multilingual machine translation (MT). Some follow-up works further improved MT performance via preference optimization, but they leave a key aspect largely underexplored: the order in which data samples are given du…

• Dynamic and Adaptive Feature Generation with LLM

  Xinhao Zhang, Jinghan Zhang, Banafsheh Rekabdar, Yuanchun Zhou, Pengfei Wang, Kunpeng Liu · 12 de enero de 2026

  The representation of feature space is a crucial environment where data points get vectorized and embedded for subsequent modeling. Thus the efficacy of machine learning (ML) algorithms is closely related to the quality of feature engineering. As one of the most important techniques, feature generat…

• Do LLMs Need Inherent Reasoning Before Reinforcement Learning? A Study in Korean Self-Correction

  Hongjin Kim, Jaewook Lee, Kiyoung Lee, Jong-hun Shin, Soojong Lim, Oh-Woog Kwon · 12 de enero de 2026

  Large Language Models (LLMs) demonstrate strong reasoning and self-correction abilities in high-resource languages like English, but their performance remains limited in low-resource languages such as Korean. In this study, we investigate whether reinforcement learning (RL) can enhance Korean reason…

• Parallel Test-Time Scaling for Latent Reasoning Models

  Runyang You, Yongqi Li, Meng Liu, Wenjie Wang, Liqiang Nie, Wenjie Li · 12 de enero de 2026

  Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs), typically by sampling multiple token-based chains-of-thought in parallel and aggregating outcomes through voting or search. Recent advances in latent reasoning, where intermediate reasoning unfolds in …

• VietMix: A Naturally-Occurring Parallel Corpus and Augmentation Framework for Vietnamese-English Code-Mixed Machine Translation

  Hieu Tran, Phuong-Anh Nguyen-Le, Huy Nghiem, Quang-Nhan Nguyen, Wei Ai, Marine Carpuat · 12 de enero de 2026

  Machine translation (MT) systems universally degrade when faced with code-mixed text. This problem is more acute for low-resource languages that lack dedicated parallel corpora. This work directly addresses this gap for Vietnamese-English, a language context characterized by challenges including ort…

• RingSQL: Generating Synthetic Data with Schema-Independent Templates for Text-to-SQL Reasoning Models

  Marko Sterbentz, Kevin Cushing, Cameron Barrie, Kristian J. Hammond · 12 de enero de 2026

  Recent advances in text-to-SQL systems have been driven by larger models and improved datasets, yet progress is still limited by the scarcity of high-quality training data. Manual data creation is expensive, and existing synthetic methods trade off reliability and scalability. Template-based approac…

• Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models

  Gen Li, Changxiao Cai · 9 de enero de 2026

  Diffusion models have emerged as a powerful paradigm for modern generative modeling, demonstrating strong potential for large language models (LLMs). Unlike conventional autoregressive (AR) models that generate tokens sequentially, diffusion models allow for parallel sampling, offering a promising p…

• XGrammar 2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs

  Linzhang Li, Yixin Dong, Guanjie Wang, Ziyi Xu, Alexander Jiang, Tianqi Chen · 9 de enero de 2026

  Modern LLM agents are required to handle increasingly complex structured generation tasks, such as tool calling and conditional structured generation. These tasks are significantly more dynamic than predefined structures, posing new challenges to the current structured generation engines. In this pa…

• Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop

  Yaxuan Wang, Zhongteng Cai, Yujia Bao, Xueru Zhang, Yang Liu · 9 de enero de 2026

  The rapid advancement of large language models (LLMs) has led to growing interest in using synthetic data to train future models. However, this creates a self-consuming retraining loop, where models are trained on their own outputs and may cause performance drops and induce emerging biases. In real-…

• Excess Description Length of Learning Generalizable Predictors

  Elizabeth Donoway, Hailey Joren, Fabien Roger, Jan Leike · 9 de enero de 2026

  Understanding whether fine-tuning elicits latent capabilities or teaches new ones is a fundamental question for language model evaluation and safety. We develop a formal information-theoretic framework for quantifying how much predictive structure fine-tuning extracts from the train dataset and writ…

• SampoNLP: A Self-Referential Toolkit for Morphological Analysis of Subword Tokenizers

  Iaroslav Chelombitko, Ekaterina Chelombitko, Aleksey Komissarov · 9 de enero de 2026

  The quality of subword tokenization is critical for Large Language Models, yet evaluating tokenizers for morphologically rich Uralic languages is hampered by the lack of clean morpheme lexicons. We introduce SampoNLP, a corpus-free toolkit for morphological lexicon creation using MDL-inspired Self…

• HGMF: A Hierarchical Gaussian Mixture Framework for Scalable Tool Invocation within the Model Context Protocol

  Wenpeng Xing, Zhipeng Chen, Changting Lin, Meng Han · 9 de enero de 2026

  Invoking external tools enables Large Language Models (LLMs) to perform complex, real-world tasks, yet selecting the correct tool from large, hierarchically-structured libraries remains a significant challenge. The limited context windows of LLMs and noise from irrelevant options often lead to low s…

• From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs

  Kaiyuan Deng, Hangyu Zheng, Minghai Qing, Kunxiong Zhu, Gen Li, Yang Xiao, Lan Emily Zhang, Linke Guo, Bo Hui, Yanzhi Wang, Geng Yuan, Gagan Agrawal, Wei Niu, Xiaolong Ma · 8 de enero de 2026

  Deploying models, especially large language models (LLMs), is becoming increasingly attractive to a broader user base, including those without specialized expertise. However, due to the resource constraints of certain hardware, maintaining high accuracy with larger model while meeting the hardware r…

• Policy-Guided Search on Tree-of-Thoughts for Efficient Problem Solving with Bounded Language Model Queries

  Sumedh Pendurkar, Guni Sharon · 8 de enero de 2026

  Recent studies explored integrating state-space search algorithms with Language Models (LM) to perform look-ahead on the token generation process, the ''Tree-of-Thoughts'' (ToT), generated by LMs, thereby improving performance on problem-solving tasks. However, the affiliated search algorithms often…

• Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models

  Chenxi Zhou, Pengfei Cao, Jiang Li, Bohan Yu, Jinyu Ye, Jun Zhao, Kang Liu · 8 de enero de 2026

  Post-Training Quantization (PTQ) is a critical strategy for efficient Large Language Models (LLMs) deployment. However, existing scaling laws primarily focus on general performance, overlooking crucial fine-grained factors and how quantization differentially impacts diverse knowledge capabilities. T…

• Compositional Discrete Latent Code for High Fidelity, Productive Diffusion Models

  Samuel Lavoie, Michael Noukhovitch, Aaron Courville · 7 de enero de 2026

  We argue that diffusion models' success in modeling complex distributions is, for the most part, coming from their input conditioning. This paper investigates the representation used to condition diffusion models from the perspective that ideal representations should improve sample fidelity, be easy…

• TWIST: Training-free and Label-free Short Text Clustering through Iterative Vector Updating with LLMs

  I-Fan Lin, Faegheh Hasibi, Suzan Verberne · 7 de enero de 2026

  In this paper, we propose a training-free and label-free method for short text clustering that can be used on top of any existing embedder. In the context of customer-facing chatbots, companies are dealing with large amounts of user utterances that need to be clustered according to their intent. In …

• Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languages

  Tewodros Kederalah Idris, Prasenjit Mitra, Roald Eiselen · 7 de enero de 2026

  Cross-lingual transfer is essential for building NLP systems for low-resource African languages, but practitioners lack reliable methods for selecting source languages. We systematically evaluate five embedding similarity metrics across 816 transfer experiments spanning three NLP tasks, three Africa…

• VocabTailor: Dynamic Vocabulary Selection for Downstream Tasks in Small Language Models

  Hanling Zhang, Yayu Zhou, Tongcheng Fang, Zhihang Yuan, Guohao Dai, Wanli Ouyang, Yu Wang · 7 de enero de 2026

  Small Language Models (SLMs) provide computational advantages in resource-constrained environments, yet memory limitations remain a critical bottleneck for edge device deployment. A substantial portion of SLMs' memory footprint stems from vocabulary-related components, particularly embeddings and la…

• CaTS-Bench: Can Language Models Describe Time Series?

  Luca Zhou, Pratham Yashwante, Marshall Fisher, Alessio Sampieri, Zihao Zhou, Fabio Galasso, Rose Yu · 7 de enero de 2026

  Time series captioning, the task of describing time series in natural language, requires numeric and temporal reasoning, trend interpretation, and contextual understanding. Existing benchmarks, however, often rely on fully synthetic or generic captions, and typically neglect metadata and visual repr…

• MemHunter: Automated and Verifiable Memorization Detection at Dataset-scale in LLMs

  Zhenpeng Wu, Jian Lou, Zibin Zheng, Chuan Chen · 7 de enero de 2026

  Large language models (LLMs) have been shown to memorize and reproduce content from their training data, raising significant privacy concerns, especially with web-scale datasets. Existing methods for detecting memorization are primarily sample-specific, relying on manually crafted or discretely opti…

• Learning from Prompt itself: the Hierarchical Attribution Prompt Optimization

  Dongyu Chen, Jian Ma, Xianpeng Zhang, Lei Zhang, Haonan Lu, Chen Chen, Chuangchuang Wang, Kai Tang · 7 de enero de 2026

  Optimization is fundamental across numerous disciplines, typically following an iterative process of refining an initial solution to enhance performance. This principle is equally critical in prompt engineering, where designing effective prompts for large language models constitutes a complex optimi…

• Self-Speculative Biased Decoding for Faster Re-Translation

  Linxiao Zeng, Haoyun Deng, Kangyuan Shu, Shizhen Wang · 6 de enero de 2026

  Large language models achieve strong machine translation quality but incur high inference cost and latency, posing challenges for simultaneous translation. Re-translation provides a practical solution for off-the-shelf LLMs by repeatedly regenerating the target output as the source input grows, but …

• When to Ponder: Adaptive Compute Allocation for Code Generation via Test-Time Training

  Gihyeon Sim · 6 de enero de 2026

  Large language models apply uniform computation to all inputs, regardless of difficulty. We propose PonderTTT, a gating strategy using the TTT layer's self-supervised reconstruction loss to selectively trigger Test-Time Training (TTT) updates. The gating decision itself is training-free--requiring n…

• FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness

  Hossam Amer, Maryam Dialameh, Hossein Rajabzadeh, Walid Ahmed, Weiwei Zhang, Yang Liu · 6 de enero de 2026

  Scaling training compute, measured in FLOPs, has long been shown to improve the accuracy of large language models, yet training remains resource-intensive. Prior work shows that increasing test-time compute (TTC)-for example through iterative sampling-can allow smaller models to rival or surpass muc…