Power Systems and Technologies

Neueste Paper

• CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters

  Jungseob Lee, Sugyeong Eo · 2. Oktober 2026

  Speculative decoding accelerates large language model inference by drafting future tokens cheaply and verifying them with the target model in parallel. Block drafters score a whole block of future tokens in one forward pass, yet standard decoding verifies only the top-scoring chain and discards the …

• Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts

  Di He, Pengxiang Li, Da Chang, Qingyan Meng, Lu Yin, Shiwei Liu · 2. Oktober 2026

  Looped Transformers introduce recurrent depth as a new scaling axis for LLMs: by repeatedly applying shared Transformer blocks, they increase effective depth without increasing parameter count. However, the benefits of looping remain unclear for large MoE LLMs under FLOPs-matched comparisons. The ma…

• DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair

  Zhuoyu Wang, Junnan Huang, Xinyu Chen · 2. Oktober 2026

  Parallel drafting reduces the drafting overhead of speculative decoding for large language models (LLMs), but its gains remain limited by the accepted prefix length. Even when the correct token is present in the candidate pool, a single early selection error prevents subsequent predictions from bein…

• SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

  Hankun Lin, Patrick Pynadath, Ruqi Zhang · 1. Oktober 2026

  Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeoff between the quality and cost of the draft. Early-exit methods produce drafts cheaply by terminating computation at intermediate layers, but forgo the…

• Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits

  Gaurav Agarwal, Ashish Garg, Isha Singhal · 1. Oktober 2026

  Concurrent autoregressive inference creates a fundamental interference problem: prefilling a newly arrived long prompt can delay tokens for requests that are already decoding. Fixed prefill chunks reduce this interference, but the best chunk size depends on the model, hardware, load, and latency obj…

• DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification

  Rongjian Chen, Minxian Xu, Zhengxin Fang, Kejiang Ye, Chengzhong Xu · 30. September 2026

  Growing large language model applications demand efficient inference. At high concurrency, block-diffusion speculative decoding suffers from verification padding, rejected candidates, and incompatibility between variable prefixes and fixed-shape graphs. Uniform truncation sacrifices acceptable token…

• LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

  Hao-Yuan He, Peng-Fei Liu, Si Shen, Ming Li · 30. September 2026

  Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context grows, existing state-of-the-art drafters become increasingly expensive, eroding the very efficiency advantage they are designed to provide. We argue …

• Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving

  Yi Xu, Ehsan K. Ardestani, Wenyin Fu, Martin Schatz, Krishna Malladi, Zhan Shu, Adnan Aziz, Shobhit Kanaujia, Ajit Mathews, Chunqiang Tang · 24. September 2026

  As serving capacity demand surpasses that of training, serving efficiency becomes increasingly important. Prefill-decode (P/D) disaggregation improves serving efficiency through specialization and isolation of the two phases. These benefits rest on a static partitioning. Phase demand, however, is no…

• PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models

  Xiaocheng Lu, Shuhan Guo, Ziyue Ma, Jie Zhang, Jian Liu, Jingcai Guo, Haoxuan Che, Song Guo · 23. September 2026

  Diffusion language models (dLLMs), such as LLaDA and Dream, have become competitive with autoregressive (AR) LLMs in generation quality while supporting native parallel decoding. A standard acceleration strategy is block-wise decoding, where each forward pass predicts a block of length B and commits…

• TSS: Target-Side Sparsification for Speculative Decoding in Domain-Specific Large Language Models

  Haibo Hu, Lianming Huang, Qiao Li, Nan Guan, Chun Jason Xue · 23. September 2026

  Speculative decoding accelerates large language model inference through collaboration between a lightweight draft model and a target verifier. Existing methods mainly improve the draft side, while the target model is typically kept dense and unchanged. We show that, under domain-specific inference, …

• Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

  Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia, Yutaka Watanobe, Fang Liu · 23. September 2026

  Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latency, energy consumption, and GPU memory demand as context length scales. Existing soft-compression methods either lack query-guided memory selection a…

• H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache

  Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn, Reed Meyerson, Zhenting Qi, Tianyu Wu, Eldar Kurtic, Minlan Yu, Alexandre Marques · 22. September 2026

  Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification by the target model. Recent block diffusion drafters further reduce drafting latency by predicting multiple tokens in parallel. However, existing bloc…

• Acceptance-Aware Draft Model Training for Speculative Decoding

  Tianhua Xia, Mugilan Ganesan, Yifei Feng, Haiyu Wang, Maximilian Egger, Sai Qian Zhang · 22. September 2026

  Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are verified by the target model in a single forward pass. Its speedup is largely determined by the acceptance length, yet existing draft-model training…

• GDN Tree-Scan: Served Tree Verification for Recurrent-Hybrid Language Models

  Zhiyuan Ma · 22. September 2026

  Tree speculative decoding verifies multiple candidate continuations in one target forward pass. For attention-only transformers, the verifier mainly needs an ancestry mask. Recurrent-hybrid language models break this assumption: a candidate row must also carry the recurrent state that native sequent…

• WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models

  Hyeongju Ha, Jae-Joon Kim · 22. September 2026

  Looped language models repeatedly apply a weight-shared block to increase effective depth without increasing parameter count, but the resulting T sequential recurrent-block calls per generated token substantially increase decoding latency. To address the issue, we introduce Wavefront Decoding (WFD),…

• Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

  Changxu Liu, Zhaogeng Li · 22. September 2026

  Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follows a single candidate chain, tree-structured speculation retains multiple branches from shared prefixes; under the same budget, this broader coverag…

• Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

  Yanxiao Liu, Sicheng Wan, Zhan Gao, Deniz G\"und\"uz · 21. September 2026

  Large language models (LLMs) have achieved state-of-the-art performance across a wide range of tasks, motivating two important aspects of deployment: inference efficiency and output provenance, which can be tackled by speculative sampling and watermarking, respectively. However, recent works have sh…

• SpecQuant: Speculative Decoding with Multi-Parent Quantization for Adaptive LLM Inference

  Harish KB, Jagadeeswaran M, Pradheep P, Yuvanesh S, Sivakumar T · 21. September 2026

  Running large language models (LLMs) locally continues to be limited by restrictions of compute and memory on consumer hardware. The popular acceleration technologies, such as quantization, speculative decoding, and adaptive inferencing, offer substantial speed boosts but usually necessitate retrain…

• RheoSampling: Resolving the One-Hot Dilemma in Stochastic Dynamic-Tree Speculative Decoding

  Qiao Hu, Yepeng Weng, Bo Zhang, Takehisa Yairi · 21. September 2026

  Speculative decoding accelerates LLM inference by drafting multiple tokens in parallel, with tree-based methods further improving efficiency through hierarchical structures. Dynamic-tree methods such as EAGLE-3 perform well under greedy decoding via deterministic top-K expansion and global pruning. …

• To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals

  Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik, Lior Wolf, Itamar Zimerman · 18. September 2026

  Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff between two drafting strategies: neural drafting and context-based copying. Neural drafts (e.g., EAGLE3) provide robust performance across diverse text se…

• LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

  SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han · 16. September 2026

  Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurrent depths. However, they incur higher decoding latency than standard Transformer models of comparable parameter size because shared weights are access…

• Carryover Drafting: Recycling Rejected States for Speculative Decoding

  Jahyun Koo, Sunghyeon Woo, Jaeeun Kil, Jeongtae Lee, Sungjae Lee, Kyomin Jung, Minsub Kim · 15. September 2026

  Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept several tokens. By construction, verification computes representations for both accepted and rejected tokens. Yet, conventional drafters retain only the re…

• PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

  Weisi Yang, Stephen Xia · 10. September 2026

  Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased privacy, personalization, and reduced latency. However, LLMs have heavy computational requirements, which are difficult for resource-constrained mobile a…

• Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding

  Fengxiang Bie, Yuqing Jian, Yifan Yu, Zhongzhu Zhou, Zelei Shao, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu, Tianyi Zhang · 10. September 2026

  Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target model, and their acceptance rate collapses under workload shifts. This is a striking inversion of modern LLM development,…

• X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding

  Jaeduk Lee, Wan Choi · 10. September 2026

  This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device small language model (SLM) drafts candidate tokens and a server LLM verifies them. Existing CoSD methods assume a shared vocabulary between the SLM an…