Tensor decomposition and applications

Latest papers

• Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison

  Rahul Krishnan, Volker Schulz · 24 September 2026

  The key-value (KV) cache of autoregressive transformers can be viewed as a fourth-order tensor spanning attention heads, tokens, features, and grouped layers. We measure the singular-value spectra of all four mode unfoldings on Mistral-7B-v0.3 and LLaMA-2-13B and compare four standard tensor decompo…

• Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning

  Marius Willner, Marco Trenti, Dirk Lebiedz · 23 September 2026

  Tree tensor networks (TTNs) are widely used in low-rank approximation and quantum many-body simulation. In this work, we present a formal analysis of the quotient geometry underlying the TTN parameter space. Our framework allows for arbitrary horizontal distributions, and we develop efficient first-…

• Guaranteed Low-Rank Tensor Recovery from Modewise Measurements via Normalized Block-Weighted Riemannian Gradient Descent

  Yushi Zhou, Feng Zhang · 22 September 2026

  We consider the recovery of low-multilinear-rank tensors from linear measurements and propose an adaptive block-weighted modewise Riemannian gradient descent method. The method combines memory-efficient modewise measurements with a normalized adaptive weighting strategy for the core and factor compo…

• Nonnegative Matrix Factorization in the Component-Wise L1 Norm for Sparse Data

  Giovanni Seraghiti, K\'evin Dubrulle, Arnaud Vandaele, Nicolas Gillis · 21 September 2026

  Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, X, by the product of two nonnegative factors, WH, where W has r columns and H has r rows. In this paper, we consider NMF using the component-wise L1 norm as the error measure (L1-NMF), which is suited for data corrupted by hea…

• MultiHU-TD: Multifeature Hyperspectral Unmixing Based on Tensor Decomposition

  Mohamad Jouni, Mauro Dalla Mura, Lucas Drumetz, Pierre Comon · 21 September 2026

  Hyperspectral unmixing allows representing mixed pixels as a set of pure materials weighted by their abundances. Spectral features alone are often insufficient, so it is common to rely on other features of the scene. Matrix models become insufficient when the hyperspectral image (HSI) is represented…

• Benign Loss Landscapes Can Coexist with Worst-Case Hardness

  Zach Furman, Stephan W\"aldchen, Yangda Bei, Liam Hodgkinson · 14 September 2026

  Deep neural networks are expressive enough to contain worst-case targets that can be evaluated in polynomial time but cannot be learned in polynomial time by gradient descent. For practical tasks they nonetheless learn well, raising the question of what non-generic structure of real-world targets en…

• Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery

  Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang, Xi-Le Zhao, Deyu Meng · 14 September 2026

  Recently, tensor decompositions are prevalent for multi-dimensional image representation, which learn the instance-specific structure of each image from scratch. However, tensor decompositions neglect the common structure across different images, leading to limited semantic modeling capability, high…

• RunningTensor: Generalizing Linear Attention to Higher-Order Recurrent States

  Luca Herranz-Celotti, Vincent Guigue · 14 September 2026

  Linear attention and state-space models provide linear-time sequence modeling, but their recurrent memory remains a second-order tensor (a matrix), limiting the order of interactions that can be represented in the state. We introduce the RunningTensor, which generalizes this memory to an order-$o$ t…

• Identifiability of Nonnegative Tensor Decompositions via Positive Scattering

  Haoming Wang, Ming Yuan · 11 September 2026

  Identifiability of tensor decompositions is often established through linear-algebraic conditions on the factor families. For nonnegative decompositions, however, positivity provides additional information that is not captured by dimension and independence alone: nonnegative terms cannot cancel, and…

• Semi-Tensor Product-Based Multi-Term Randomized T-SVD and Its Visual Applications

  Xingchen Xiao (School of Mathematics and Statistics, Southwest University, Chongqing, China), Feng Zhang (School of Mathematics and Statistics, Southwest University, Chongqing, China), Wenjin Qin (School of Mathematics and Statistics, Southwest University, Chongqing, China), Jianjun Wang (School of Mathematics and Statistics, Southwest University, Chongqing, China) · 11 September 2026

  Tensor singular value decomposition (T-SVD), which is built upon the tensor-tensor product (t-product), has emerged as a powerful tool for processing high-dimensional visual data such as color images and videos. However, the standard t-product imposes strict dimensional compatibility constraints. Al…

• A Block Tensor Train Burer-Monteiro Framework for Low-Rank Quantum State Tomography

  Shakir Showkat Sofi, Charlotte Vermeylen, Fatemeh Mohammadi, Lieven De Lathauwer · 10 September 2026

  Quantum state tomography is a fundamental technique for estimating the state of a quantum system from measured data and plays a crucial role in evaluating the performance of quantum devices. However, standard estimation methods become computationally prohibitive as the system size increases due to t…

• Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics

  Will Houser, Vanja Dukic, David M. Bortz · 10 September 2026

  In recent years, weak-form methods have made significant advances in data-driven discovery of dynamical systems. However, in high-dimensional settings, current techniques can prove expensive in both computation and memory. In this work, we introduce TT-WSINDy, which combines techniques of the Multid…

• Tensor network representations of discrete maximum entropy distributions via mean polytopes

  Alex Goessmann, Martin Eigel · 9 September 2026

  We present tensor network representations for discrete maximum entropy distributions under expectation constraints. To this end, we introduce Computation-Activation Networks (CompActNets), a tensor network architecture that subsumes exponential families. By leveraging the geometry of the convex poly…

• Linear Algebra Foundations of Efficient Attention: A Phase Reversal in Rank Collapse Under SVD Compression

  Anjaneya Teja Sarma Kalvakolanu · 9 September 2026

  Linear algebra provides the framework of concepts (matrix rank, singular value decomposition (SVD), and eigendecomposition) that modern artificial intelligence employs to encode, compress, and propagate information through neural networks. This paper unifies fourteen separate peer-reviewed works ana…

• Parameterised graph theory for tensor networks: entanglement rerouting, structural simplification, and agnostic tomography

  Matthias C. Caro, Natalie McHugh, Sergii Strelchuk · 4 September 2026

  Parameterised graph theory studies how the complexity of graph-theoretic problems depends on structural parameters of the input graph. This perspective has proved useful in analysing tensor-network simulation (Markov and Shi, 2008). Its implications for tensor-network representations and tomography …

• Coupled Tensor-Tensor Completion Method with Applications in Drug Repurposing

  Maryam Bagherian, Albert Hung, Ivo Dinov, Joshua Welch · 4 September 2026

  Many biomedical challenges can be posed as tensor completion problems where the observed entries of a multidimensional array (a tensor) are used to impute the missing values. In such settings, incorporating side information about the modes of the tensor, such as gene-gene similarity, can significant…

• Stochastic Optimization of Tree Tensor Networks

  Marius Willner, Maximilian Scharf, Andr\'e Uschmajew, Timo Felser, Marco Trenti · 2 September 2026

  Tensor networks, originally developed for quantum many-body physics, are promising models for machine learning. We derive stochastic Riemannian optimizers for tree tensor networks (TTNs) on both their parameter and quotient manifolds, including adaptive and learning-rate-free schemes suitable for mi…

• Higher Structures in Deep Learning

  Michael L. Roberts, Carlos Zapata Carratal\'a. Nicholas J. Cooper, Lijun Chen, Fran\c{c}ois G. Meyer, Danna Gurari · 2 September 2026

  We provide an expository introduction on the importance of higher-arity tensor operations to deep learning. Then, we conduct a novel empirical investigation of higher-arity phenomenon in trained neural networks, introduce a hypergraphical generalization of the multilayer perceptron, and explore conn…

• A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability

  Zhang Enyan, R. Thomas McCoy · 1 September 2026

  A wide range of methods have been proposed for interpreting language models, delivering important insights into their inner workings. However, different methods and their resulting insights stand in relative isolation: what could the underlying structure of language models be, such that they give ri…

• Equivariant Covariance Tensors: Guaranteed SPD Uncertainty for Tensor-Valued Geometric Learning

  Ruihan Liu, Yu Ji, Jianbo Yu, Shifu Yan, Qingchao Jiang · 26 August 2026

  Tensor-valued prediction is fundamental to geometric deep learning, yet uncertainty quantification (UQ) for such outputs remains an open challenge. While E(3)-equivariant neural networks excel at point estimates, they lack rigorous confidence measures. We focus on symmetric rank-2 tensor prediction,…

• On the Choice of Tensor Estimation for Corner Detection, Optical Flow and Denoising

  Freddie {\AA}str\"om, Michael Felsberg · 25 August 2026

  Many image processing methods such as corner detection, optical flow and iterative enhancement make use of image tensors. Generally, these tensors are estimated using the structure tensor. In this work we show that the gradient energy tensor can be used as an alternative to the structure tensor in s…

• Tensor Field Models

  Alexander Strunk, Roland Assam · 20 August 2026

  This paper introduces Tensor Field Models (TFMs), realization-level Mathematical Structures in which a learned Operator maps a product of admissible component-section families to a prescribed family of time-dependent tangent sections on a Generative State Manifold. Analytic and dynamical restriction…

• Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

  Xiao Wang, Tomohiro Hashizume, Pia Siegl, Dieter Jaksch · 19 August 2026

  Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for…

• Improving the matrix multiplication exponent with modern optimization and AlphaEvolve

  Emilien Dupont, Marvin Eisenberger, Borislav Kozlovskii, Abbas Mehrabian, Francisco J. R. Ruiz, Abigail See, Renfei Zhou, Josh Alman, Virginia Vassilevska Williams, Matej Balog · 18 August 2026

  The current best bounds on the matrix multiplication exponent $ω$ are obtained through a refinement of the laser method called combination loss analysis (Duan et al., 2022; Williams et al., 2024; Alman et al., 2025). In this note, we address the optimization problem at the core of this approach and …

• High-Dimensional Nonparametric Change-Point Detection via Low-Rank Degree-Three Density Projection

  Guoqing Zhang, Zhaixin Chen · 18 August 2026

  Distributional changes can be invisible to means and covariances yet appear in skewness, asymmetric interactions, or other third-order structure. We develop a nonparametric change-point method that retains every degree-at-most-three coefficient of a density while avoiding direct density estimation. …