Natural Language Processing Techniques

Últimos artículos

• Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning

  Shambhavi Krishna, Atharva Naik, Chaitali Agarwal, Sudharshan Govindan, Taesung Lee, Haw-Shiuan Chang · 11 de noviembre de 2025

  Large language models are increasingly deployed across diverse applications. This often includes tasks LLMs have not encountered during training. This implies that enumerating and obtaining the high-quality training data for all tasks is infeasible. Thus, we often need to rely on transfer learning u…

• RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services

  Fei Zhao, Chonggang Lu, Haofu Qian, Fangcheng Shi, Zijie Meng, Jianzhao Huang, Xu Tang, Zheyong Xie, Zheyu Ye, Zhe Xu, Yao Hu, Shaosheng Cao · 11 de noviembre de 2025

  As a key medium for human interaction and information exchange, social networking services (SNS) pose unique challenges for large language models (LLMs): heterogeneous workloads, fast-shifting norms and slang, and multilingual, culturally diverse corpora that induce sharp distribution shift. Supervi…

• Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving

  Hui Zeng, Daming Zhao, Pengfei Yang, Wenxuan Hou, Tianyang Zheng, Hui Li, Weiye Ji, Jidong Zhai · 11 de noviembre de 2025

  Generative reasoning with large language models (LLMs) often involves long decoding sequences, leading to substantial memory and latency overheads from accumulating key-value (KV) caches. While existing KV compression methods primarily focus on reducing prefill memory from long input sequences, they…

• P3-LLM: An Integrated NPU-PIM Accelerator for LLM Inference Using Hybrid Numerical Formats

  Yuzong Chen, Chao Fang, Xilai Dai, Yuheng Wu, Thierry Tambe, Marian Verhelst, Mohamed S. Abdelfattah · 11 de noviembre de 2025

  The substantial memory bandwidth and computational demand of large language models (LLMs) present critical challenges for efficient inference. To tackle this, the literature has explored heterogeneous systems that combine neural processing units (NPUs) with DRAM-based processing-in-memory (PIM) for …

• HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor

  Zihui Wu, Haichang Gao, Jiacheng Luo, Zhaoxiang Liu · 11 de noviembre de 2025

  Large Language Models (LLMs) commonly rely on explicit refusal prefixes for safety, making them vulnerable to prefix injection attacks. We introduce HumorReject, a novel data-driven approach that reimagines LLM safety by decoupling it from refusal prefixes through humor as an indirect refusal strate…

• Preparation of Fractal-Inspired Computational Architectures for Advanced Large Language Model Analysis

  Yash Mittal, Dmitry Ignatov, Radu Timofte · 11 de noviembre de 2025

  It introduces FractalNet, a fractal-inspired computational architectures for advanced large language model analysis that mainly challenges model diversity on a large scale in an efficient manner. The new set-up involves a template-driven generator, runner, and evaluation framework that, through syst…

• JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models

  Haibo Jin, Leyang Hu, Xinnuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, Haohan Wang · 11 de noviembre de 2025

  The rapid evolution of artificial intelligence (AI) through developments in Large Language Models (LLMs) and Vision-Language Models (VLMs) has brought significant advancements across various technological domains. While these models enhance capabilities in natural language processing and visual inte…

• Benchmarking Large Language Models with Integer Sequence Generation Tasks

  Daniel O'Malley, Manish Bhattarai, Nishath Rajiv Ranasinghe, Erick Draayer, Javier Santos · 11 de noviembre de 2025

  We present a novel benchmark designed to rigorously evaluate the capabilities of large language models (LLMs) in mathematical reasoning and algorithmic code synthesis tasks. The benchmark comprises integer sequence generation tasks sourced from the Online Encyclopedia of Integer Sequences (OEIS), te…

• iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use

  Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Wu Ning, Yutai Hou, Xu Huang, Duyu Tang, Dandan Tu, Bing Qin, Ting Liu · 10 de noviembre de 2025

  Augmenting large language models (LLMs) with external tools is a promising approach to enhance their capabilities, especially for complex tasks. Synthesizing tool-use data through real-world simulations is an effective way to achieve this. However, our investigation reveals that training gains signi…

• Attention and Compression is all you need for Controllably Efficient Language Models

  Jatin Prakash, Aahlad Puli, Rajesh Ranganath · 10 de noviembre de 2025

  The quadratic cost of attention in transformers motivated the development of efficient approaches: namely sparse and sliding window attention, convolutions and linear attention. Although these approaches result in impressive reductions in compute and memory, they often trade-off with quality, specif…

• Iterative Layer-wise Distillation for Efficient Compression of Large Language Models

  Grigory Kovalev, Mikhail Tikhomirov · 10 de noviembre de 2025

  This work investigates distillation methods for large language models (LLMs) with the goal of developing compact models that preserve high performance. Several existing approaches are reviewed, with a discussion of their respective strengths and limitations. An improved method based on the ShortGPT …

• HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts

  Neil He, Rishabh Anand, Hiren Madhu, Ali Maatouk, Smita Krishnaswamy, Leandros Tassiulas, Menglin Yang, Rex Ying · 7 de noviembre de 2025

  Large language models (LLMs) have shown great success in text modeling tasks across domains. However, natural language exhibits inherent semantic hierarchies and nuanced geometric structure, which current LLMs do not capture completely owing to their reliance on Euclidean operations. Recent studies …

• KGGen: Extracting Knowledge Graphs from Plain Text with Language Models

  Belinda Mo, Kyssen Yu, Joshua Kazdan, Joan Cabezas, Proud Mpala, Lisa Yu, Chris Cundy, Charilaos Kanatsoulis, Sanmi Koyejo · 7 de noviembre de 2025

  Recent interest in building foundation models for KGs has highlighted a fundamental challenge: knowledge-graph data is relatively scarce. The best-known KGs are primarily human-labeled, created by pattern-matching, or extracted using early NLP techniques. While human-generated KGs are in short suppl…

• Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training

  William Merrill, Shane Arora, Dirk Groeneveld, Hannaneh Hajishirzi · 7 de noviembre de 2025

  The right batch size is important when training language models at scale: a large batch size is necessary for fast training, but a batch size that is too large will harm token efficiency. To navigate this tradeoff, McCandlish et al. (2018) suggest that a critical batch size (CBS), below which traini…

• OMPILOT: Harnessing Transformer Models for Auto Parallelization to Shared Memory Computing Paradigms

  Arijit Bhattacharjee, Ali TehraniJamsaz, Le Chen, Niranjan Hasabnis, Mihai Capota, Nesreen Ahmed, Ali Jannesari · 7 de noviembre de 2025

  Recent advances in large language models (LLMs) have significantly accelerated progress in code translation, enabling more accurate and efficient transformation across programming languages. While originally developed for natural language processing, LLMs have shown strong capabilities in modeling p…

• Training Large Language Models To Reason In Parallel With Global Forking Tokens

  Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan · 7 de noviembre de 2025

  Although LLMs have demonstrated improved performance by scaling parallel test-time compute, doing so relies on generating reasoning paths that are both diverse and accurate. For challenging problems, the forking tokens that trigger diverse yet correct reasoning modes are typically deep in the sampli…

• Optimizing Reasoning Efficiency through Prompt Difficulty Prediction

  Bo Zhao, Berkcan Kapusuzoglu, Kartik Balasubramaniam, Sambit Sahu, Supriyo Chakraborty, Genta Indra Winata · 7 de noviembre de 2025

  Reasoning language models perform well on complex tasks but are costly to deploy due to their size and long reasoning traces. We propose a routing approach that assigns each problem to the smallest model likely to solve it, reducing compute without sacrificing accuracy. Using intermediate representa…

• Training Optimal Large Diffusion Language Models

  Jinjie Ni, Qian Liu, Chao Du, Longxu Dou, Hang Yan, Zili Wang, Tianyu Pang, Michael Qizhe Shieh · 6 de noviembre de 2025

  We introduce Quokka, the first systematic scaling law for diffusion language models (DLMs), encompassing both compute-constrained and data-constrained regimes, and studying the key modeling and optimization designs. Quokka is a good friend of Chinchilla and provides wider scopes. We hope the results…

• Dense SAE Latents Are Features, Not Bugs

  Xiaoqing Sun, Alessandro Stolfo, Joshua Engels, Ben Wu, Senthooran Rajamanoharan, Mrinmaya Sachan, Max Tegmark · 6 de noviembre de 2025

  Sparse autoencoders (SAEs) are designed to extract interpretable features from language models by enforcing a sparsity constraint. Ideally, training an SAE would yield latents that are both sparse and semantically meaningful. However, many SAE latents activate frequently (i.e., are \emph{dense}), ra…

• Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Models

  Nghia Bui, Guergana Savova, Lijing Wang · 6 de noviembre de 2025

  The impact of random seeds in fine-tuning large language models (LLMs) has been largely overlooked despite its potential influence on model performance.In this study, we systematically evaluate the effects of random seeds on LLMs using the GLUE and SuperGLUE benchmarks. We analyze the macro-level im…

• VoiceAgentBench: Are Voice Assistants ready for agentic tasks?

  Dhruv Jain, Harshit Shukla, Gautam Rajeev, Ashish Kulkarni, Chandra Khatri, Shubham Agarwal · 6 de noviembre de 2025

  Large-scale Speech Language Models (SpeechLMs) have enabled voice assistants capable of understanding natural spoken queries and performing complex tasks. However, existing speech benchmarks primarily focus on isolated capabilities such as transcription, or question-answering, and do not systematica…

• BanglaSTEM: A Parallel Corpus for Technical Domain Bangla-English Translation

  Kazi Reyazul Hasan, Mubasshira Musarrat, A. B. M. Alim Al Islam, Muhammad Abdullah Adnan · 6 de noviembre de 2025

  Large language models work well for technical problem solving in English but perform poorly when the same questions are asked in Bangla. A simple solution would be to translate Bangla questions into English first and then use these models. However, existing Bangla-English translation systems struggl…

• Training Language Models to Reason Efficiently

  Daman Arora, Andrea Zanette · 5 de noviembre de 2025

  Scaling model size and training data has led to great advances in the performance of Large Language Models (LLMs). However, the diminishing returns of this approach necessitate alternative methods to improve model capabilities, particularly in tasks requiring advanced reasoning. Large reasoning mode…

• New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models

  Julia Wunderle, Anton Ehrmanntraut, Jan Pfister, Fotis Jannidis, Andreas Hotho · 4 de noviembre de 2025

  Encoders remain essential for efficient German NLP and NLU scenarios despite the rise of decoder-only LLMs. This work studies two routes to high-quality German encoders under identical data and training constraints: 1) training from scratch and 2) converting decoders via LLM2Vec. We introduce two re…

• Exploring the Hidden Capacity of LLMs for One-Step Text Generation

  Gleb Mezentsev, Ivan Oseledets · 4 de noviembre de 2025

  A recent study showed that large language models (LLMs) can reconstruct surprisingly long texts - up to thousands of tokens - via autoregressive generation from just one trained input embedding. In this work, we explore whether autoregressive decoding is essential for such reconstruction. We show th…