Parallel Computing and Optimization Techniques

Neueste Paper

• Modernizing Amdahl's Law: How AI Scaling Laws Shape Computer Architecture

  Chien-Ping Lu · 10. April 2026

  Classical Amdahl's Law conceptualized the limit of speedup for an era of fixed serial-parallel decomposition and homogeneous replication. Modern heterogeneous systems need a different conceptual framework: constrained resources must be allocated across heterogeneous hardware while workloads themselv…

• ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache

  Shao Wang, Rui Ren, Lin Gui · 9. April 2026

  The serving paradigm of large language models (LLMs) is rapidly shifting towards complex multi-agent workflows where specialized agents collaborate over massive shared contexts. While Low-Rank Adaptation (LoRA) enables the efficient co-hosting of these specialized agents on a single base model, it i…

• Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start

  Xueshen Liu, Yongji Wu, Yuncheng Yao, Danyang Zhuo, Ion Stoica, Z. Morley Mao · 9. April 2026

  Modern LLM service providers increasingly rely on autoscaling and parallelism reconfiguration to respond to rapidly changing workloads, but cold-start latency remains a major bottleneck. While recent systems have reduced model weight loading to seconds, CUDA graph capture still takes tens of seconds…

• Splats under Pressure: Exploring Performance-Energy Trade-offs in Real-Time 3D Gaussian Splatting under Constrained GPU Budgets

  Muhammad Fahim Tajwar, Arthur Wuhrlin, Bhojan Anand · 9. April 2026

  We investigate the feasibility of real-time 3D Gaussian Splatting (3DGS) rasterisation on edge clients with varying Gaussian splat counts and GPU computational budgets. Instead of evaluating multiple physical devices, we adopt an emulation-based approach that approximates different GPU capability ti…

• AutoLALA: Automatic Loop Algebraic Locality Analysis for AI and HPC Kernels

  Yifan Zhu, Yekai Pan, Yanghui Wu, Chen Ding · 8. April 2026

  Data movement is the primary bottleneck in modern computing systems. For loop-based programs common in high-performance computing (HPC) and AI workloads, including matrix multiplication, tensor contraction, stencil computation, and einsum operations, the cost of moving data through the memory hierar…

• Comparative Characterization of KV Cache Management Strategies for LLM Inference

  Oteo Mamo, Olga Kogiou, Hyunjin Yi, Weikuan Yu · 8. April 2026

  Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value vectors at each layer. These caches are essential to minimize redundant computation during autoregressive token generation, lowering computational complexity …

• Attention Editing: A Versatile Framework for Cross-Architecture Attention Conversion

  Zhen Cheng, Hao-Bo Yang, Wan-Yi Huang, Jin-Long Li · 8. April 2026

  Key-Value (KV) cache memory and bandwidth increasingly dominate large language model inference cost in long-context and long-generation regimes. Architectures such as multi-head latent attention (MLA) and hybrid sliding-window attention (SWA) can alleviate this bound, but integrating them into exist…

• Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression

  Longsheng Zhou, Yu Shen · 8. April 2026

  Modern deployment often requires trading accuracy for efficiency under tight CPU and memory constraints, yet common compression proxies such as parameter count or FLOPs do not reliably predict wall-clock inference time. In particular, unstructured sparsity can reduce model storage while failing to a…

• LP-GEMM: Integrating Layout Propagation into GEMM Operations

  C\'esar Guedes Carneiro, Lucas Alvarenga, Guido Araujo, Sandro Rigo · 7. April 2026

  In Scientific Computing and modern Machine Learning (ML) workloads, sequences of dependent General Matrix Multiplications (GEMMs) often dominate execution time. While state-of-the-art BLAS libraries aggressively optimize individual GEMM calls, they remain constrained by the BLAS API, which requires …

• When GPUs Fail Quietly: Observability-Aware Early Warning Beyond Numeric Telemetry

  Michael Bidollahkhani, Freja Nordsiek, Julian M. Kunkel · 7. April 2026

  GPU nodes are central to modern HPC and AI workloads, yet many failures do not manifest as immediate hard faults. While some instabilities emerge gradually as weak thermal or efficiency drift, a significant class occurs abruptly with little or no numeric precursor. In these detachment-class failures…

• AXELRAM: Quantize Once, Never Dequantize

  Yasushi Nishida · 6. April 2026

  We propose AXELRAM, a smart SRAM macro architecture that computes attention scores directly from quantized KV cache indices without dequantization. The key enabler is a design-time fixed codebook: orthogonal-transform-based quantization concentrates each coordinate's distribution to N(0,1/d), so the…

• A Practical Two-Stage Framework for GPU Resource and Power Prediction in Heterogeneous HPC Systems

  Beste Oztop, Dhruva Kulkarni, Zhengji Zhao, Ayse Kivilcim Coskun, Kadidia Konate · 3. April 2026

  Efficient utilization of GPU resources and power has become critical with the growing demand for GPUs in high-performance computing (HPC). In this paper, we analyze GPU utilization and GPU memory utilization, as well as the power consumption of the Vienna ab initio Simulation Package (VASP), using t…

• CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

  Tara Saba, Anne Ouyang, Xujie Si, Fan Long · 3. April 2026

  High-performance GPU kernels are critical to modern machine learning systems, yet developing efficient implementations remains a challenging, expert-driven process due to the tight coupling between algorithmic structure, memory hierarchy usage, and hardware-specific optimizations. Recent work has ex…

• MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

  Zifei Xu, Sayeh Sharify, Hesham Mostafa · 2. April 2026

  Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, where a single model is trained to be robus…

• SISA: A Scale-In Systolic Array for GEMM Acceleration

  Luigi Altamura, Alessio Cicero, Mateo V\'azquez Maceiras, Mohammad Ali Maleki, Pedro Trancoso · 1. April 2026

  The currently dominant AI/ML workloads, such as Large Language Models (LLMs), rely on the efficient execution of General Matrix-Matrix Multiplication (GEMM) operations. Thus, most systems are equipped with dedicated matrix hardware accelerators based on square Systolic Arrays (SAs) of Processing Ele…

• StepCache: Step-Level Reuse with Lightweight Verification and Selective Patching for LLM Serving

  Azam Nouri · 1. April 2026

  We address LLM serving workloads where repeated requests share a common solution structure but differ in localized constraints, such as output schema, variable names, or numeric constants. Prior caching approaches typically reuse either full responses (semantic caching) or model-internal KV/prefix s…

• Time is Not Compute: Scaling Laws for Wall-Clock Constrained Training on Consumer GPUs

  Yi Liu · 1. April 2026

  Scaling laws relate model quality to compute budget (FLOPs), but practitioners face wall-clock time constraints, not compute budgets. We study optimal model sizing under fixed time budgets from 5 minutes to 24 hours on consumer GPUs (RTX 4090). Across 70+ runs spanning 50M--1031M parameters, we find…

• Improving Efficiency of GPU Kernel Optimization Agents using a Domain-Specific Language and Speed-of-Light Guidance

  Siva Kumar Sastry Hari, Vignesh Balaji, Sana Damani, Qijing Huang, Christos Kozyrakis · 1. April 2026

  Optimizing GPU kernels with LLM agents is an iterative process over a large design space. Every candidate must be generated, compiled, validated, and profiled, so fewer trials will save both runtime and cost. We make two key observations. First, the abstraction level that agents operate at is import…

• KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Study

  Suraj Ranganath, Vaishak Menon, Anish Patnaik · 31. März 2026

  Self-forcing video generation extends a short-horizon video model to longer rollouts by repeatedly feeding generated content back in as context. This scaling path immediately exposes a systems bottleneck: the key-value (KV) cache grows with rollout length, so longer videos require not only better ge…

• Modernizing Amdahl's Law: How AI Scaling Laws Shape Computer Architecture

  Chien-Ping Lu · 31. März 2026

  Classical Amdahl's Law quantifies the limit of speedup under a fixed serial-parallel decomposition and homogeneous replication. Modern systems instead allocate constrained resources across heterogeneous hardware while the workload itself changes: some stages become effectively bounded, whereas other…

• ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference

  Qiuyang Zhang, Kai Zhou, Ding Tang, Kai Lu, Cheng Li, Zhenyu Yang, Peng Xu, Jiguang Wan · 31. März 2026

  Large language models encounter critical GPU memory capacity constraints during long-context inference, where KV cache memory consumption severely limits decode batch sizes. While existing research has explored offloading KV cache to DRAM, these approaches either demand frequent GPU-CPU data transfe…

• IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression

  Zhongping Ji · 31. März 2026

  Orthogonal feature decorrelation is effective for low-bit online vector quantization, but dense random orthogonal transforms incur prohibitive $O(d^2)$ storage and compute. RotorQuant reduces this cost with blockwise $3$D Clifford rotors, yet the resulting $3$D partition is poorly aligned with moder…

• GPU-Accelerated Optimization of Transformer-Based Neural Networks for Real-Time Inference

  Soutrik Mukherjee, Sangwhan Cha · 31. März 2026

  This paper presents the design and evaluation of a GPU-accelerated inference pipeline for transformer models using NVIDIA TensorRT with mixed-precision optimization. We evaluate BERT-base (110M parameters) and GPT-2 (124M parameters) across batch sizes from 1 to 32 and sequence lengths from 32 to 51…

• RSR-core: A High-Performance Engine for Low-Bit Matrix-Vector Multiplication

  Mohsen Dehghankar, Abolfazl Asudeh · 31. März 2026

  Matrix-vector multiplication is a fundamental building block in neural networks, vector databases, and large language models, particularly during inference. As a result, efficient matrix-vector multiplication engines directly translate into more efficient inference. Recent work has explored low-bit …

• CoDec: Prefix-Shared Decoding Kernel for LLMs

  Zhibin Wang, Rui Ning, Chao Fang, Zhonghui Zhang, Xi Lin, Shaobo Ma, Mo Zhou, Xue Li, Zhongfeng Wang, Chengying Huan, Rong Gu, Kun Yang, Guihai Chen, Sheng Zhong, Chen Tian · 31. März 2026

  Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critical bottleneck with increasing context lengths, is a memory-intensive process requiring heavy memory access on the key-va…