Explainable Artificial Intelligence (XAI)

Latest papers

• SEAM: Global consistency beyond local accuracy in scientific machine learning

  Gnankan Landry Regis N'guessan, Bum Jun Kim · 7 August 2026

  Scientific machine learning commonly validates models at the level of a subdomain, a benchmark split, or an explanation for one prediction. Yet such local checks cannot establish whether the resulting explanations can be assembled into one globally admissible explanation. We introduce Scientific Exp…

• When Do Corrective Features Help? An Agent for Corrective Feature Discovery on Black-Box Forecasters

  Fangxin Wang, Ziyi Zhang, Diyi Zhuang, Langzhou He, Shiyu Wang, Baichuan Mo, Philip S. Yu · 7 August 2026

  Frozen pretrained forecasters often fail in structured, recurring ways that are costly to repair through fine-tuning. We study corrective feature discovery: mining interpretable features of a frozen forecaster's residual to drive a lightweight post-hoc corrector. Prior automated feature engineering …

• Why the Third Axis Is Freedom

  Michael Timothy Bennett · 7 August 2026

  In generative training, a model produces an output and is penalised for its difference from an example. With one output per comparison, a model that produces one common answer can outperform a model retaining a broader repertoire. Explorative Modeling (XM) produces $K$ outputs per comparison and upd…

• Challenges in Evaluating Explanation Methods for Static and Evolving Data

  Jerzy Stefanowski · 7 August 2026

  This paper addresses the limitations of Explainable Artificial Intelligence (XAI) with respect to insufficient evaluation. They are illustrated through the DetoxAI image recognition system for bias detection and concept unlearning. Then, an example of a human-grounded evaluation of methods for expla…

• Measuring Explainer Stability via Attribution Separability

  Eddie Conti, \'Alvaro Parafita, Axel Brando · 5 August 2026

  Attribution methods (AMs) assign an importance score to each feature and are widely adopted to explain black-box models. However, most methods can produce variable attribution scores due to stochastic components in their definition. In this paper, we propose a distribution-based framework to capture…

• Explainable AI for the EU Right to Explanation: A Systematic Review of the Law-XAI Translation Gap

  Benjamin Fresz, Elena Dubovitskaya, Marco F. Huber · 5 August 2026

  When algorithms make or influence consequential decisions---about loan eligibility, hiring, or healthcare---EU law grants affected individuals a Right to Explanation. Yet whether (and how) Explainable AI (XAI) can satisfy this right in practice remains poorly understood, with direct implications for…

• LLMs Can Annotate Attribution Graphs

  Ameen Patel, Max Zhang, Nathan Hu · 5 August 2026

  Circuit tracing is an exciting technique for revealing the internal computation of language models, but it requires a time-intensive manual step of grouping individual features or MLP neurons into supernodes. We present a simple pipeline for automating this step: directly presenting feature descript…

• Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs

  Vahidin Hasic, Chao Wang, Luis C. Garcia-Peraza-Herrera, David Watson, Senka Krivic · 4 August 2026

  Multimodal large language models (MLLMs) increasingly support high-stakes decision making by combining complementary information from images and text. While existing explainability methods identify influential image regions or text tokens, they cannot answer a fundamental question: which modality dr…

• Crushing the Evidence: A Dual-Penalty Evasion Framework for Fooling White-Box Explainable AI Auditors

  Niraj Kumar, Harsh Kasyap · 4 August 2026

  Post-hoc model explainers such as LIME, SHAP, and Integrated Gradients are widely deployed to audit models in high-stakes sensitive domains, including finance, healthcare, and social welfare. This ensures the model's transparency and acceptability. However, a few studies have examined potential atta…

• Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions

  Zachary Izzo · 4 August 2026

  In this paper, we study the connection between an LLM's output distribution and the data used to train it. Specifically, we study the degree to which an LLM's next-token distribution agrees with the empirical next-token distribution (ENTD) given the context in the training data. The ENTD is an appea…

• Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterization

  Tyler Ashoff, Jordan Rodu · 3 August 2026

  Modern opaque AI models prize performance over interpretability, which makes testing difficult. However, formal statistical tests conducted on a model's embedding space can provide robust characterizations of semantic structure, concept separation, and knowledge graph alignment. Model developers wou…

• A Generalized-Bayes Perspective on Counterfactual Explanations: Posterior-Based Decision-Making and Evaluation

  Keita Kinjo · 3 August 2026

  Counterfactual explanations (CEs) enhance the interpretability of machine learning models by identifying the smallest change to an input required to obtain a desired output. Although CEs are conventionally formulated as a distance-minimization problem, the theoretical basis of this formulation has r…

• A Human-Centered Validation of the Explainability-Performance Coefficient

  Christian Oliva, Luis F. Lago-Fern\'andez · 3 August 2026

  The rapid adoption of deep learning models in high-risk domains has intensified the need for trustworthy Explainable Artificial Intelligence (XAI). However, objectively evaluating explanation fidelity and aligning XAI metrics with human-centered understanding remain critical open challenges. In this…

• StaQ: a Finite Memory Approach to Discrete Action Policy Mirror Descent

  Alex Davey, Alena Shilova, Brahim Driss, Riad Akrour · 3 August 2026

  In Reinforcement Learning (RL), regularization with a Kullback-Leibler divergence that penalizes large deviations between successive policies has emerged as a popular tool both in theory and practice. This family of algorithms, often referred to as Policy Mirror Descent (PMD), has the property of av…

• Automorphism-Induced Non-Canonicity in Top-k Explanations of Graph Neural Networks

  Xin Xu, Siru Tao, Kaizhen Tan · 30 July 2026

  A gradient-based GNN explainer given a molecule with two chemically equivalent nitro groups assigns them attribution scores that are equal to the last bit. It cannot do otherwise: message passing is exactly permutation equivariant, so any automorphism of the input leaves every attribution invariant.…

• Entangled by Design: Spurious Intra-Variable Signal Routing in Tabular In-Context Learners

  Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz, Sotirios Tsaftaris · 29 July 2026

  Consider a model trained at a single hospital to predict patient recovery, where the measured feature $X$ bundles the patient's true health signal ($C$) with a systematic artefact from that hospital's equipment ($S$). Within that hospital, the artefact correlates with outcomes through unmeasured con…

• Behavior-Driven Explainability

  Caroline Dominik, Rolf Drechsler · 29 July 2026

  As system complexity has vastly increased, it has become significantly more challenging for a single person or a team to fully understand all aspects of an entire system. Particularly, this holds when considering all the different stages of a system's development life cycle, such as, e.g., design or…

• Interpretable GOHR Agents via Sparse Autoencoders

  Shiwei Tan, Yusong Zhao, Weiyi Qin, Wentian Wang, Jacob Feldman, Lazaros K. Gallos, Paul B. Kantor, Vladimir Menkov, Hao Wang · 29 July 2026

  A central challenge in interpreting learned decision-making systems is to determine whether their internal representations contain concepts that help explain their behavior. We report interpretability experiments for a tokenized autoregressive Transformer agent in the Game of Hidden Rules (GOHR). We…

• On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy

  Aline Mangold, Juliane Zietz, Susanne Weinhold, Sebastian Pannasch · 29 July 2026

  As AI becomes more common in everyday living, there is an increasing demand for intelligent systems that are both performant and understandable. Explainable AI (XAI) systems aim to provide comprehensible explanations of decisions and predictions. At present, however, evaluation processes are rather …

• TaylorPODA: A Taylor Expansion-Based Method to Improve Post-Hoc Attributions for Opaque Models

  Yuchi Tang, I\~naki Esnaola, George Panoutsos · 29 July 2026

  Post-hoc model-agnostic local attribution (LA) methods have been widely adopted to explain opaque AI models by quantifying feature-wise contributions. However, many existing methods rely on heuristic or only partially justified attribution mechanisms, while the quality of attribution itself is often…

• Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

  Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty, Iryna Gurevych, Subhabrata Dutta · 28 July 2026

  The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior. Features with clear activation descriptions may have weak or unexpected causal effects; steering can vary across prompts or oppose the intended directi…

• Variable Importance Identification Through Lazy Training for Binary Classification

  Anand Singh, Luke Pennella, Eshan Kabir, Xiaoxi Shen · 28 July 2026

  Deep neural networks have been widely used in many applications (e.g., computer vision and natural language processing); however, understanding their explainability remains a challenging task. Recently, substantial research has been devoted to improving the explainability of deep neural networks, wi…

• proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference

  Alexandra N. M. Darmon, Deeksha Sinha, Steve Wilkins-Reeves, Caner Gocmen · 28 July 2026

  Proxy outcomes (such as short-term behavioral signals, model predictions, or surrogate endpoints) are frequently used in place of primary outcomes that are too slow to mature, rare, or challenging to measure directly. But valid inference on a proxy does not guarantee valid inference on the primary e…

• Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification

  Nils Gumpfer, Michael Guckert, Samuel Sossalla, Birgit A{\ss}mus, Jennifer Hannig · 28 July 2026

  Explainable AI (XAI) is used to assess whether artificial intelligence models rely on meaningful patterns, yet explanations that appear plausible for individual predictions may systematically misrepresent model behavior. This is particularly problematic in medicine, where models may rely on irreleva…

• Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations

  Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe, M. Sohel Rahman · 28 July 2026

  Post-training quantization (PTQ) has become a practical solution for deploying deep learning models on resource-constrained edge devices by compressing high-precision floating-point weights into low-precision representations without requiring retraining. Past research has demonstrated that quantizat…