Model Reduction and Neural Networks

Derniers papiers

• Mechanistic Interpretability with Sparse Autoencoder Neural Operators

  Bahareh Tolooshams, Ailsa Shen, Anima Anandkumar · 24 février 2026

  We introduce sparse autoencoder neural operators (SAE-NOs), a new class of sparse autoencoders that operate directly in infinite-dimensional function spaces. We generalize the linear representation hypothesis to a functional representation hypothesis, enabling concept learning beyond vector-valued r…

• Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelines

  Siavash Khodakarami, Vivek Oommen, Nazanin Ahmadi Daryakenari, Maxim Beekenkamp, George Em Karniadakis · 24 février 2026

  Solving partial differential equations (PDEs) by neural networks as well as Kolmogorov-Arnold Networks (KANs), including physics-informed neural networks (PINNs), physics-informed KANs (PIKANs), and neural operators, are known to exhibit spectral bias, whereby low-frequency components of the solutio…

• Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction

  Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi, Chang Wei, Yuchen Fan, Yew-Soon Ong · 24 février 2026

  Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Alg…

• Unlearning Noise in PINNs: A Selective Pruning Framework for PDE Inverse Problems

  Yongsheng Chen, Yong Chen, Wei Guo, Xinghui Zhong · 24 février 2026

  Physics-informed neural networks (PINNs) provide a promising framework for solving inverse problems governed by partial differential equations (PDEs) by integrating observational data and physical constraints in a unified optimization objective. However, the ill-posed nature of PDE inverse problems …

• Weak-Form Evolutionary Kolmogorov-Arnold Networks for Solving Partial Differential Equations

  Bongseok Kim, Jiahao Zhang, Guang Lin · 24 février 2026

  Partial differential equations (PDEs) form a central component of scientific computing. Among recent advances in deep learning, evolutionary neural networks have been developed to successively capture the temporal dynamics of time-dependent PDEs via parameter evolution. The parameter updates are obt…

• Global Low-Rank, Local Full-Rank: The Holographic Encoding of Learned Algorithms

  Yongzhong Xu · 24 février 2026

  Grokking -- the abrupt transition from memorization to generalization after extended training -- has been linked to the emergence of low-dimensional structure in learning dynamics. Yet neural network parameters inhabit extremely high-dimensional spaces. How can a low-dimensional learning process pro…

• Transformers for dynamical systems learn transfer operators in-context

  Anthony Bao, Jeffrey Lai, William Gilpin · 24 février 2026

  Large-scale foundation models for scientific machine learning adapt to physical settings unseen during training, such as zero-shot transfer between turbulent scales. This phenomenon, in-context learning, challenges conventional understanding of learning and adaptation in physical systems. Here, we s…

• SGNO: Spectral Generator Neural Operators for Stable Long Horizon PDE Rollouts

  Jiayi Li, Zhaonan Wang, Flora D. Salim · 24 février 2026

  Neural operators provide fast PDE surrogates and often generalize across parameters and resolutions. However, in the short train long test setting, autoregressive rollouts can become unstable. This typically happens for two reasons: one step errors accumulate over time, and high frequency components…

• Regularity of Second-Order Elliptic PDEs in Spectral Barron Spaces

  Ziang Chen, Liqiang Huang, Mengxuan Yang, Shengxuan Zhou · 24 février 2026

  We establish a regularity theorem for second-order elliptic PDEs on $\mathbb{R}^{d}$ in spectral Barron spaces. Under mild ellipticity and smallness assumptions, the solution gains two additional orders of Barron regularity. As a corollary, we identify a class of PDEs whose solutions can be approxim…

• Vid2Sid: Videos Can Help Close the Sim2Real Gap

  Kevin Qiu, Yu Zhang, Marek Cygan, Josie Hughes · 24 février 2026

  Calibrating a robot simulator's physics parameters (friction, damping, material stiffness) to match real hardware is often done by hand or with black-box optimizers that reduce error but cannot explain which physical discrepancies drive the error. When sensing is limited to external cameras, the pro…

• Who Said Neural Networks Aren't Linear?

  Nimrod Berman, Assaf Hallak, Assaf Shocher · 23 février 2026

  Neural networks are famously nonlinear. However, linearity is defined relative to a pair of vector spaces, $f:X \to Y$. Leveraging the algebraic concept of transport of structure, we propose a method to explicitly identify non-standard vector spaces where a neural network acts as a linear operator. …

• Solving and learning advective multiscale Darcian dynamics with the Neural Basis Method

  Yuhe Wang, Min Wang · 23 février 2026

  Physics-governed models are increasingly paired with machine learning for accelerated predictions, yet most "physics--informed" formulations treat the governing equations as a penalty loss whose scale and meaning are set by heuristic balancing. This blurs operator structure, thereby confounding solu…

• NIMMGen: Learning Neural-Integrated Mechanistic Digital Twins with LLMs

  Zihan Guan, Rituparna Datta, Mengxuan Hu, Shunshun Liu, Aiying Zhang, Prasanna Balachandran, Sheng Li, Anil Vullikanti · 23 février 2026

  Mechanistic models encode scientific knowledge about dynamical systems and are widely used in downstream scientific and policy applications. Recent work has explored LLM-based agentic frameworks to automatically construct mechanistic models from data; however, existing problem settings substantially…

• Deepmechanics

  Abhay Shinde, Aryan Amit Barsainyan, Jose Siguenza, Ankita Vaishnobi Bisoi, Rakshit Kr. Singh, Bharath Ramsundar · 23 février 2026

  Physics-informed deep learning models have emerged as powerful tools for learning dynamical systems. These models directly encode physical principles into network architectures. However, systematic benchmarking of these approaches across diverse physical phenomena remains limited, particularly in co…

• Parameter-Efficient Domain Adaptation of Physics-Informed Self-Attention based GNNs for AC Power Flow Prediction

  Redwanul Karim (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), Changhun Kim (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), Timon Conrad (Institute of Electrical Energy Systems, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Germany), Nora Gourmelon (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), Julian Oelhaf (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), David Riebesel (Institute of Electrical Energy Systems, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Germany), Tom\'as Arias-Vergara (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), Andreas Maier (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany), Johann J\"ager (Institute of Electrical Energy Systems, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Germany), Siming Bayer (Pattern Recognition Lab, Friedrich-Alexander-Universit\"at Erlangen-N\"urnberg, Erlangen, Germany) · 23 février 2026

  Accurate AC-PF prediction under domain shift is critical when models trained on medium-voltage (MV) grids are deployed on high-voltage (HV) networks. Existing physics-informed graph neural solvers typically rely on full fine-tuning for cross-regime transfer, incurring high retraining cost and offeri…

• Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver

  Pietro Sittoni, Emanuele Zangrando, Angelo A. Casulli, Nicola Guglielmi, Francesco Tudisco · 23 février 2026

  Deep learning-based methods have shown remarkable effectiveness in solving PDEs, largely due to their ability to enable fast simulations once trained. However, despite the availability of high-performance computing infrastructure, many critical applications remain constrained by the substantial comp…

• PHAST: Port-Hamiltonian Architecture for Structured Temporal Dynamics Forecasting

  Shubham Bhardwaj, Chandrajit Bajaj · 23 février 2026

  Real physical systems are dissipative -- a pendulum slows, a circuit loses charge to heat -- and forecasting their dynamics from partial observations is a central challenge in scientific machine learning. We address the \emph{position-only} (q-only) problem: given only generalized positions~$q_t$ at…

• Causality by Abstraction: Symbolic Rule Learning in Multivariate Timeseries with Large Language Models

  Preetom Biswas, Giulia Pedrielli, K. Sel\c{c}uk Candan · 23 février 2026

  Inferring causal relations in timeseries data with delayed effects is a fundamental challenge, especially when the underlying system exhibits complex dynamics that cannot be captured by simple functional mappings. Traditional approaches often fail to produce generalized and interpretable explanation…

• Generating adversarial inputs for a graph neural network model of AC power flow

  Robert Parker · 23 février 2026

  This work formulates and solves optimization problems to generate input points that yield high errors between a neural network's predicted AC power flow solution and solutions to the AC power flow equations. We demonstrate this capability on an instance of the CANOS-PF graph neural network model, as…

• The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning

  Mojtaba Sahraee-Ardakan, Mauricio Delbracio, Peyman Milanfar · 23 février 2026

  Autonomous (noise-agnostic) generative models, such as Equilibrium Matching and blind diffusion, challenge the standard paradigm by learning a single, time-invariant vector field that operates without explicit noise-level conditioning. While recent work suggests that high-dimensional concentration a…

• Point-DeepONet: Predicting Nonlinear Fields on Non-Parametric Geometries under Variable Load Conditions

  Jangseop Park, Namwoo Kang · 20 février 2026

  Nonlinear structural analyses in engineering often require extensive finite element simulations, limiting their applicability in design optimization and real-time control. Conventional deep learning surrogates often struggle with complex, non-parametric three-dimensional (3D) geometries and directio…

• AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing

  Jianda Du, Youran Sun, Haizhao Yang · 20 février 2026

  PDEs are central to scientific and engineering modeling, yet designing accurate numerical solvers typically requires substantial mathematical expertise and manual tuning. Recent neural network-based approaches improve flexibility but often demand high computational cost and suffer from limited inter…

• FEKAN: Feature-Enriched Kolmogorov-Arnold Networks

  Sidharth S. Menon, Ameya D. Jagtap · 19 février 2026

  Kolmogorov-Arnold Networks (KANs) have recently emerged as a compelling alternative to multilayer perceptrons, offering enhanced interpretability via functional decomposition. However, existing KAN architectures, including spline-, wavelet-, radial-basis variants, etc., suffer from high computationa…

• Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning

  Binghang Lu, Jiahao Zhang, Guang Lin · 19 février 2026

  Physics-informed neural networks and neural operators often suffer from severe optimization difficulties caused by ill-conditioned gradients, multi-scale spectral behavior, and stiffness induced by physical constraints. Recently, the Muon optimizer has shown promise by performing orthogonalized upda…

• Distributed physics-informed neural networks via domain decomposition for fast flow reconstruction

  Yixiao Qian, Jiaxu Liu, Zewei Xia, Song Chen, Chao Xu, Shengze Cai · 19 février 2026

  Physics-Informed Neural Networks (PINNs) offer a powerful paradigm for flow reconstruction, seamlessly integrating sparse velocity measurements with the governing Navier-Stokes equations to recover complete velocity and latent pressure fields. However, scaling such models to large spatiotemporal dom…