Parallel Computing and Optimization Techniques

Últimos artículos

• SAGE: A Novelty Gate for Efficient Memory Evolution in Agentic LLMs

  Sijia Wang, Dhanajit Brahma, Ricardo Henao · 1 de junio de 2026

  Agentic LLMs must continuously decide whether newly extracted facts should be added, merged with existing memories, or ignored, yet prior work has focused more on retrieval and storage than on principled write-side control. We frame memory evolution as a novelty-detection problem and propose SAGE, a…

• Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended

  Aditya K Kamath, Arvind Krishnamurthy, Marco Canini, Simon Peter · 1 de junio de 2026

  Machine learning (ML) training and inference often process data sets far exceeding GPU memory capacity, forcing them to rely on PCIe for on-demand tensor transfers, causing critical transfer bottlenecks. Lossy compression has been proposed to relieve bottlenecks but introduces workload-dependent acc…

• Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation

  Haozhou Zhang · 1 de junio de 2026

  Chain-of-thought prompting and looped Transformers both give a fixed model more test-time computation, but they differ in what they remember. Chain-of-thought stores intermediate state in generated tokens that remain in the context, whereas a looped Transformer carries state through recurrent hidden…

• Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation

  Soumyadeep Jana, Sagar Nishad, Sanasam Ranbir Singh · 29 de mayo de 2026

  Key-Value (KV) cache remains a major bottleneck for deploying Large Language Models (LLMs) in long-generation tasks. Prior work often applies uniform compression across both prefill and decoding caches, but compressing the prefill cache degrades performance by corrupting critical context. While pres…

• CompilerDream: Learning a Compiler World Model for General Code Optimization

  Chaoyi Deng, Jialong Wu, Ningya Feng, Jianmin Wang, Mingsheng Long · 29 de mayo de 2026

  Effective code optimization in compilers is crucial for computer and software engineering. The success of these optimizations primarily depends on the selection and ordering of the optimization passes applied to the code. While most compilers rely on a fixed sequence of optimization passes, current …

• PassNet: Scaling Large Language Models for Graph Compiler Pass Generation

  Yiqun Liu, Yingsheng Wu, Ruqi Yang, Enrong Zheng, Honglei Qiu, Sijun He, Tai Liang, Jingjing Wu, Yuhan Zhou, Yiwei Zhang, Dongyan Chen, Weihan Yi, Xinqi Li, Siqi Bao · 29 de mayo de 2026

  Modern tensor compilers such as TorchInductor deliver substantial speedups on mainstream models, yet face a systematic performance ceiling on long-tail workloads -- our profiling shows that 43% of real-world subgraphs experience end-to-end slowdowns under default compilation. While LLMs offer a path…

• RW-TTT: Batched Serving for Request-Owned Test-Time Training State

  Jian Yang, Zhizhuo Kou, Yao Tian, Hao Zhang, Han Chen, Sirui Han, Yike Guo · 28 de mayo de 2026

  Test-time training (TTT) adapts an LLM during generation by reading and updating request-owned state, such as fast weights, low-rank deltas, or streaming learner state. This breaks batched LLM serving, which assumes shared static weights: serial execution is correct but slow, while naive batching ca…

• The Energy Blind Spot: NVIDIA's Flagship Edge AI Hardware Cannot Support Process-Level Energy Attribution

  Deepak Panigrahy, Aakash Tyagi · 28 de mayo de 2026

  Agentic AI workloads - where a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery - are being targeted for edge deployment, with NVIDIA, Dell, HP, ASUS, MSI, Acer, and Gigabyte all shipping GB10-based desktop AI systems in 2026. We recently demonstrated tha…

• Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages

  Shuoming Zhang, Qiuchu Yu, Yangyu Zhang, Ruiyuan Xu, Xiyu Shi, Guangli Li, Xiaobing Feng, Huimin Cui, Jiacheng Zhao · 28 de mayo de 2026

  LLM-based agents are increasingly used to generate GPU kernels, but they often know what optimizations to try without knowing when those optimizations are sound. We introduce KLineage, which learns this missing "when" knowledge from expert kernels: instead of relying on forward rollouts, KLineage wa…

• Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression

  Kabir Swain, Sijie Han, Daniel Karl I. Weidele, Mauro Martino, David Cox, Antonio Torralba · 28 de mayo de 2026

  We propose \textbf{Hurwitz Quaternion Multiplicative Quantization (HQMQ)}, a \textbf{calibration-free} method for KV cache compression of large language models. HQMQ treats each 4-element chunk of K or V as a quaternion and quantizes its unit direction to the \emph{product} $q_p \cdot q_s$, where $q…

• The Energy Blind Spot: NVIDIA's Flagship Edge AI Hardware Cannot Support Process-Level Energy Attribution

  Deepak Panigrahy, Aakash Tyagi · 28 de mayo de 2026

  Agentic AI workloads - where a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery - are being targeted for edge deployment, with NVIDIA, Dell, HP, ASUS, MSI, Acer, and Gigabyte all shipping GB10-based desktop AI systems in 2026. We recently demonstrated tha…

• Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training

  Yingying Cheng, Jinquan Shi, Li Zhou, Zhiyang He, Zhaoyi Sun, Fan Zhang, Jie Sun · 27 de mayo de 2026

  Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible to standard training metrics. We present a systematic study of HiF8 W8A8 QAT for OpenPangu-Embedded-1B through the lens of Delayed Tensor Scaling (DTS…

• Xe-Forge: Multi-Stage LLM-Powered Kernel Optimization for Intel GPU

  Marcin Spoczynski, Daniel Fleischer, Moshe Berchansky, Gabriela Ben-Melech Stan, Shira Guskin, Weilin Xu, Adam Siemieniuk, Alexander Heinecke · 27 de mayo de 2026

  Porting deep learning algorithms to new hardware accelerators requires developers to repeatedly apply the same low-level optimizations -- quantization, memory access coalescing, tile size tuning, and architecture-specific workarounds -- to every Triton kernel in their code-base. This manual, repetit…

• Stateful Inference for Low-Latency Multi-Agent Tool Calling

  Victor Norgren · 27 de mayo de 2026

  Multi-agent tool calling is becoming the dominant interaction pattern for LLM-based systems, yet existing inference frameworks treat each tool call as an independent request, re-processing the entire conversation from scratch even though 85-95% of the prompt is unchanged from the previous turn. We…

• Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage

  Alan Milligan, Zikun Xu, Simon Lacoste-Julien, Felix Dangel, Wu Lin · 27 de mayo de 2026

  Shampoo-based methods, such as KL-Shampoo and SOAP, have demonstrated strong performance in training neural networks and rely on QR decomposition. Because existing QR implementations require single-precision (FP32) arithmetic and remain computationally expensive, these methods become time- and memor…

• CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging

  Shiyang Li, Haoyang Chen, Mattia Fazzini, Caiwen Ding · 27 de mayo de 2026

  Debugging CUDA programs has long been challenging because failures often arise from subtle interactions among hardware behavior, compiler decisions, memory hierarchy, and asynchronous execution. More importantly, with the rapid expansion of GPU usage across scientific computing, machine learning, gr…

• EXOTIC: An Exact, Optimistic, Tree-Based Algorithm for Min-Max Optimization

  Chinmay Maheshwari, Chinmay Pimpalkhare, Debasish Chatterjee · 26 de mayo de 2026

  Min-max optimization arises in many domains such as game theory, adversarial machine learning, etc. For these problems, gradient-based methods are well understood and enjoy strong guarantees. However, in the absence of convexity or concavity, existing approaches study convergence to an approximate s…

• Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment

  Muhammad Azlan Qazi, Alexandros Iosifidis, Qi Zhang · 26 de mayo de 2026

  Distributing Transformer inference across embedded edge devices can alleviate individual memory and compute constraints, yet practical benefits on real hardware remain unclear: prior work relies largely on simulations that overlook hardware-specific communication overheads. We present a hardware pro…

• A Tabular Schedule Abstraction for Communication-Aware Evaluation of Pipeline-Parallel LLM Training

  Daniel Barley, Jonathan Leis, Benjamin Klenk, Holger Fr\"oning · 26 de mayo de 2026

  Pipeline parallelism is a key technique for distributed training of large language models because it reduces per-device parameter and activation memory. However, comparing pipeline schedules is difficult: analytical models expose structural quantities such as bubble ratios, while end-to-end hardware…

• The Model Parking Tax: Quantifying the Hidden Energy Cost of Always-On GPU Model Deployment

  Sai Sathvik Vadari · 26 de mayo de 2026

  The AI inference industry keeps models loaded in GPU memory around the clock to avoid cold-start latency, implicitly treating idle power as a fixed cost of readiness. Yet the structure of this cost has never been empirically decomposed - and never across GPU architectures. We present the first cross…

• CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

  Yubo Li, Yidi Miao · 26 de mayo de 2026

  Long-horizon LLM inference turns the key--value (KV) cache into the dominant GPU memory consumer and makes per-token attention increasingly expensive. Many common eviction policies use static recency windows or historical attention, leaving unused a signal computed on every decoding step: the model'…

• Fine-Tuning and Serving Gemma 4 31B on Google Cloud TPU: A Technical Comparison with GPU Baselines

  Jatin Kishnani, Mayank Goel, Amit Singh, Pulkit Agrawal, Sairanjan Mishra · 26 de mayo de 2026

  We present the first end-to-end demonstration of fine-tuning and serving Google's Gemma 4 31B model on TPU hardware, providing an empirical comparison of TPU and GPU platforms for large language model adaptation. Using LoRA on a Google TPU v5p-8 for training and TPU v6e-8 (Trillium) for inference, w…

• SA-Kura: An Energy-Efficient Systolic Array Accelerator for Locally-Coupled Kuramoto Drift in Diffusion Sampling

  Jeongmin Jin, Kyeongwon Lee, Mundo Jeong, Jongin Choi, Woojoo Lee · 26 de mayo de 2026

  Diffusion inference remains costly for edge deployment, yet existing accelerators focus almost exclusively on score networks because standard drift is merely a trivial linear scaling. Kuramoto orientation diffusion replaces this trivial drift with locally coupled phase interactions, improving sampli…

• ScaleAcross Explorer: Exploring Communication Optimization for Scale-Across AI Model Training

  Minghao Li, Alicia Golden, Samuel Hsia, Michael Kuchnik, Adi Gangidi, Xu Zhang, Ashmitha Jeevaraj Shetty, Zachary DeVito, Weiwei Chu, Dong He, Haoci Zhang, Yuchen Hao, Ruoming Pang, James Hongyi Zeng, Ying Zhang, Minlan Yu, Carole-Jean Wu · 26 de mayo de 2026

  The rapid scaling of large language model training requires distributing GPU resources across multiple data center buildings and regions. We refer to such paradigm as "scale-across" training. As infrastructure expands, the system design space becomes increasingly intricate, encompassing new model ar…

• Adaptive Mass-Segmented KV Compression for Long-Context Reasoning

  Junzhe Yang, Xiaoyu Shen · 25 de mayo de 2026

  The linear growth of the Key-Value (KV) cache is a critical bottleneck in long-form LLM inference. Existing KV compression methods mitigate this by evicting tokens based on importance scores. However, we show that their reliance on global Top-k selection triggers Region Wipe-out: the severe eviction…