Parallel Computing and Optimization Techniques

Latest papers

• Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment

  Hyeju Shin, Chorwon Kim, Ryangsoo Kim, Hark Yoo, Jaein Kim · 10 July 2026

  The emergence of vision language models with fewer than 3 billion parameters has accelerated the implementation of on-device multimodal intelligence. However, a detailed understanding of component-wise quantization remains a bottleneck for optimal deployment. This paper presents a systematic evaluat…

• Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

  Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han, Han Cai · 10 July 2026

  Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order of magnitude past the pretraining window, making zero-shot context e…

• CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems

  Tingkai Liu, Muralidhar Andoorveedu, Sanjoy Das, Sanjay Patel, Volodymyr Kindratenko · 10 July 2026

  The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures. However, current software still mostly treats these coherent interconnects simply as high-speed networks. Simultaneously, the demand for serving Large Language Models under lat…

• At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

  Truong Xuan Khanh · 9 July 2026

  On modular arithmetic, a network's embedding keeps compressing for tens of thousands of steps after it has already generalized. Reading effective rank at the grokking transition overstates the converged value by 3-5x on an MLP, and by 1.3-1.5x on a transformer trained to convergence; on the MLP it a…

• HiDVFS: Hierarchical Multi-Agent DVFS for Real-Time OpenMP DAG Workloads

  Mohammad Pivezhandi, Abusayeed Saifullah, Ali Jannesari · 9 July 2026

  Leakage power in multicore embedded systems now rivals dynamic power, so DVFS schedulers must respect deadlines and thermal limits, not just average makespan. Existing heuristics lack per-core, temperature-aware control and overlook the irregular execution of OpenMP DAGs. We propose HiDVFS, a genera…

• The Key to Going Linear: Analysis-Driven Transformer Linearization

  Anna Kuzina, Paul N. Whatmough, Babak Ehteshami Bejnordi · 9 July 2026

  The quadratic cost of causal self-attention severely bottlenecks long-context transformer inference. While numerous post hoc linearization pipelines exist, it is difficult to identify which components preserve model quality. This work isolates the effect of state update design in a strict frozen-bac…

• DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

  Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo, Mar Linares Tercero, Julia Barrientos, Ainhoa Miranda, Jesus Olivera · 8 July 2026

  Long-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different pr…

• Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving

  Nikita Agrawal, Ruben Mayer · 8 July 2026

  Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are difficult to compare because they were evaluated on different models, tasks, budgets, and serving stacks. This paper presents a workload-aware benchm…

• FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

  Anna C\'ordoba, Adam Puente Tercero, Nerea Angulo Hijo, Mar Linares Tercero, Julia Barrientos, Ainhoa Miranda, Jes\'us Olivera · 8 July 2026

  Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adja…

• Think Before You Grid-Search: Floor-First Triage for LLM Serving

  Yihua Liu · 8 July 2026

  LLM serving optimization typically benchmarks many configurations and reaches for heavy profilers when latency targets are missed. We argue for the reverse discipline: estimation is the analytical layer of profiling -- without it, optimization degenerates to grid search. Floor First is a residual-dr…

• GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

  Minjie Hua, Ning Wang, Peijun Yang, Kai Wang, Shiguo Lian · 7 July 2026

  OpenClaw requests are dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the context window. For this workload, with about 28k-30k input tokens and 500 output tokens per request, serving quality is governed by throughput, TTFT, …

• Elastic Gang: Per-Token Membership Change for a Hard-Barriered LLM Inference Gang Co-Scheduled with OS Processes

  Daeyeon Son · 7 July 2026

  On-device LLM decoding is a hard-barriered CPU-SIMD computation that wants every core for milliseconds per token, while the rest of the OS wants those same cores continuously. A barriered gang cannot simply be dropped into a preemptive scheduler: an unannounced departure deadlocks a barrier, and an …

• Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters

  Zonghang Li, Tao Li, Wenjiao Feng, Rongxing Xiao, Jianshu She, Hong Huang, Mohsen Guizani, Hongfang Yu, Qirong Ho, Wei Xiang, Xue Liu · 7 July 2026

  On-device inference offers privacy, offline use, and instant response, but consumer hardware restricts large language models (LLMs) to low throughput and capability. To overcome this challenge, we present prima.cpp, a distributed on-device inference system that runs 30-70B LLMs on consumer home clus…

• PLoRA: Efficient Concurrent LoRA Training for Large Language Models

  Minghao Yan, Zhuang Wang, Zhen Jia, Shivaram Venkataraman, Yida Wang · 7 July 2026

  Low-Rank Adaptation (LoRA) has gained popularity as a fine-tuning approach for Large Language Models (LLMs) due to its low resource requirements and good performance. While numerous studies have investigated ways to improve LoRA serving efficiency by serving multiple LoRAs concurrently, existing met…

• Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters

  Yoshiyuki Ootani · 7 July 2026

  Grokking -- the delayed onset of generalization long after a network has fit its training set - -is usually studied in models too large to read completely and reported from single training runs. We instead study a publicly released ~11,856-parameter Llama-style transformer (Glimmer-1-Base) on modula…

• Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning

  Saurabhsingh Rajput, Tushar Sharma · 7 July 2026

  Code models strictly prioritize functional correctness, leaving software energy efficiency as an unoptimized byproduct. Training models to generate energy-efficient code requires reproducible feedback at scale, which physical hardware measurement cannot reliably provide due to variance. In this pa…

• Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing

  Joel Klein, Rebecca Pelke, Roberto Laudani, Jan Moritz Joseph, Rainer Leupers · 7 July 2026

  Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads. However, existing ML workload partitioning approaches for CIM accelerators do not fully account for Resistive Random Access Memory (RR…

• KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

  Shen Han, Yuyang Wu, Junpu Yu, Olexandr Isayev · 7 July 2026

  Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high decoding latency and limited throughput. To address these issues, KV cache compression has emerged as a promising technique for reducing memory overhea…

• Tile-Level Activation Overlap for Efficient LLM Inference

  Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt, Jianlan YE, Chaoyin Li, Atul Gupta · 7 July 2026

  SwiGLU is the dominant MLP activation in modern large language models, yet its intermediate tensor materialization costs 9-37% of MLP execution time. We present two complementary CUTLASS-based SM90 kernels that fuse SwiGLU into GeMM at the tile level. Kernel-1 overlaps Swish computation on the Gate …

• KVpop -- Key-Value Cache Compression with Predictive Online Pruning

  Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl, David Stap, Thomas Schmied, Sebastian B\"ock, G\"unter Klambauer, Sepp Hochreiter · 7 July 2026

  Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on static heuristics or proxy scores, which poorly track future token utility and cause brittle eviction as relevance shif…

• Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention

  Siyu Ding, Mingchuan Ma, Jiabo Tong, Xingrun Xing, Ziming Wang, Guoqi Li · 7 July 2026

  Recent NVFP4 pretraining methods mainly target transformer linear layers, leaving optimizer states, optimizer arithmetic and attention underexplored in 4-bit pipelines. This critical gap blocks stable full-stack 4-bit pretraining, as the three core modules exhibit unique numerical failure patterns: …

• AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

  Sadra Saremi · 7 July 2026

  With the rise of small quantized GGUF-based language models and their increasing use for on-device inference tasks, we have seen the growing need for an approach capable of reliably delivering these models at scale even under severe memory bandwidth constraints such as those imposed by pure CPU impl…

• GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving

  Xinwei Qiang, Yifan Hu, Shixuan Sun, Jing Yang, Han Zhao, Chen Chen, Yu Feng, Jingwen Leng, Minyi Guo · 3 July 2026

  Diffusion Transformers (DiTs) have become the dominant architecture for image and video generation, creating growing demand for efficient DiT serving. Existing systems assign each request a fixed parallel configuration throughout its lifetime. However, DiT workloads exhibit substantial heterogeneity…

• Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

  Shen Han, Yuyang Wu · 3 July 2026

  Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high decoding latency and limited throughput. To address these issues, KV cache compression has emerged as a promising technique for reducing memory overhea…

• KernelSight-LM: A Kernel-Level LLM Inference Simulator

  Xiteng Yao, Taeho Kim, Hengzhi Pei, Xinle Liu, Kyle Ulrich, Leonard Lausen, Ashish Khetan, Xiang Song, George Karypis, Martin Herbordt · 3 July 2026

  As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to meet cost and latency targets. However, the end-to-end behavior of LLMs couples serving-layer policies with low-level GPU…