Explainable Artificial Intelligence (XAI)

Derniers papiers

• Distilling Tabular Foundation Models for Structured Health Data

  Aditya Tanna, Nassim Bouarour, Mohamed Bouadi, Vinay Kumar Sankarapu, Pratinav Seth · 19 mai 2026

  Tabular foundation models (TFMs) achieve strong performance on health datasets, but their inference cost and infrastructure requirements limit practical use. We study whether their predictive behavior can be transferred to lightweight tabular models through knowledge distillation. Since in-context T…

• Learning Quantifiable Visual Explanations Without Ground-Truth

  Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny, Dimosthenis Karatzas · 19 mai 2026

  Explainable AI (XAI) techniques are increasingly important for the validation and responsible use of modern deep learning models, but are difficult to evaluate due to the lack of good ground-truth to compare against. We propose a framework that serves as a quantifiable metric for the quality of XAI …

• Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs

  Xiaoyu Xu, Xiang Yue, Yang Liu, Qingqing Ye, Huadi Zheng, Peizhao Hu, Minxin Du, Haibo Hu · 19 mai 2026

  Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity. We show that these metrics can be misleading, as models can appear to forget while their original behavior is easily restored through…

• State-of-the-Art Claims Require State-of-the-Art Evidence

  YongKyung Oh · 19 mai 2026

  State-of-the-Art (SOTA) claims pervade Artificial Intelligence (AI) and Machine Learning (ML) research. These claims rest on benchmark evaluations, where models are ranked by aggregate scores across tasks. Public benchmarks or leaderboards are the most visible instance, but the same structure appear…

• From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG

  Changmin Lee, Jaemin Kim, Taesik Gong · 19 mai 2026

  With the rapid emergence of personal AI agents based on Large Language Models (LLMs), implementing them on-device has become essential for privacy and responsiveness. To handle the inherently personal and context-dependent nature of real-world requests, such agents must ground their generation in de…

• Generalized Functional ANOVA in Closed-Form: A Unified View of Additive Explanations

  Baptiste Ferrere, Nicolas Bousquet, Fabrice Gamboa, Jean-Michel Loubes · 19 mai 2026

  The functional ANOVA, or Hoeffding decomposition, provides a principled framework for interpretability by decomposing a model prediction into main effects and higher-order interactions. For independent inputs, this classical decomposition is explicit. It is closely connected to SHAP values, generali…

• Uncertainty Quantification as a Principled Foundation for Explainable Artificial Intelligence: A Case Study of Counterfactual Explanations

  Kacper Sokol, Santo M. A. R. Thies, Eyke H\"ullermeier · 19 mai 2026

  In this paper we argue that, to its detriment, transparency research overlooks many foundational concepts of artificial intelligence. As an illustrating example we focus on uncertainty quantification in the context of counterfactual explainability, demonstrating that its broader adoption could addre…

• Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy

  Adarsh Kumarappan, Ananya Mujoo · 19 mai 2026

  LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagreement at rates we term yield, a vulnerability widely attributed to RLHF-induced sycophancy. We test this attribution across four model families and find it largely wrong: pretrained base models exhibit…

• Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations

  Narges Babadi, Hadis Karimipour · 19 mai 2026

  Explanation mechanisms are increasingly used to support transparency and trust in vision-language models (VLMs), particularly in settings where model decisions require human oversight. However, the robustness of these explanations remains insufficiently understood. In this work, we investigate wheth…

• AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps

  Chia-Ying Hsieh, Hsin-Yuan Fang, Chun-Shu Wei · 19 mai 2026

  Post-hoc saliency methods are widely used to interpret deep neural networks, but their faithfulness is difficult to evaluate reliably. Existing evaluations mask features according to saliency-induced feature ordering and measure performance degradation, but this degradation can be confounded by the …

• SwordBench: Evaluating Orthogonality of Steering Image Representations

  Vladimir Zaigrajew, Dawid Pludowski, Hubert Baniecki, Przemyslaw Biecek · 19 mai 2026

  Steering or intervening on model representations at inference time to correct predictions is essential for AI interpretability and safety, yet existing evaluation protocols are limited to ambiguous language modeling tasks. To address this gap, we introduce SwordBench, a benchmark for steering image …

• Concise and Logically Consistent Conformal Sets for Neuro-Symbolic Concept-Based Models

  Samuele Bortolotti, Emanuele Marconato, Andrea Pugnana, Andrea Passerini, Stefano Teso · 19 mai 2026

  Neuro-Symbolic Concept-based Models (NeSy-CBMs) are a family of architectures that integrate neural networks with symbolic reasoning for enhanced reliability in high-stakes applications. They work by first extracting high-level concepts from the input and then inferring a task label from these compa…

• OpenJarvis: Personal AI, On Personal Devices

  Jon Saad-Falcon, Avanika Narayan, Robby Manihani, Tanvir Bhathal, Herumb Shandilya, Hakki Orhun Akengin, Gabriel Bo, Andrew Park, Matthew Hart, Caia Costello, Chuan Li, Christopher R\'e, Azalia Mirhoseini · 19 mai 2026

  Personal AI stacks, like OpenClaw and Hermes Agent, are becoming central to daily work, yet they route nearly every query (often over sensitive local data) to cloud-hosted frontier models. Replacing frontier models with local models inside existing stacks does not work: swapping Claude Opus 4.6 for …

• Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation

  Kirscher Tristan (ICube,DKFZ), Bujotzek Markus (DKFZ), Kirchhoff Yannick (DKFZ), Rokuss Maximilian (DKFZ), Isensee Fabian (DKFZ), Kahl Kim-Celine (DKFZ), Kovacs Balint (DKFZ), Maier-Hein Klaus (DKFZ) · 19 mai 2026

  Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreeme…

• UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models

  Hyunju Kang, Geonhee Han, Hogun Park · 19 mai 2026

  Node representation learning, such as Graph Neural Networks (GNNs), has emerged as a pivotal method in machine learning. The demand for reliable explanation generation surges, yet unsupervised models remain underexplored. To bridge this gap, we introduce a method for generating counterfactual (CF) e…

• Are Sparse Autoencoder Benchmarks Reliable?

  David Chanin · 19 mai 2026

  Sparse autoencoders (SAEs) are a core interpretability tool for large language models, and progress on SAE architectures depends on benchmarks that reliably distinguish better SAEs from worse ones. We audit the SAE quality metrics in SAEBench, the de-facto standard SAE evaluation suite, through thre…

• Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training

  Chenlu Ye, Zhou Yu, Ziji Zhang, Hao Chen, Narayanan Sadagopan, Jing Huang, Tong Zhang, Anurag Beniwal · 19 mai 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) improves final-answer accuracy on reasoning tasks, but it does not reliably improve reasoning quality. Because outcome rewards only assess final answers, they also reward spurious successes: flawed reasoning can still receive maximal reward when …

• Truthful Calibration Errors for Multi-Class Prediction

  Yuxuan Lu, Yifan Wu, Jason Hartline, Lunjia Hu · 19 mai 2026

  Calibrated predictions are useful because their numerical values can be interpreted as probabilities. Calibration errors are therefore widely used to evaluate, compare, and tune probabilistic predictors. Recently, Haghtalab et al. (2024) introduced an additional requirement for such measures: truthf…

• Agent Bazaar: Enabling Economic Alignment in Multi-Agent Marketplaces

  Seth Karten, Cameron Crow, Chi Jin · 19 mai 2026

  The deployment of Large Language Models (LLMs) as autonomous economic agents introduces systemic risks that extend beyond individual capability failures. As agents transition to directly interacting with marketplaces, their collective behavior can amplify volatility and mask deception at scale. We i…

• Judge Circuits

  Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia, Qianli Wang, Van Bach Nguyen, Aaron Louis Eidt, Christopher Ebert, Wojciech Samek, Jing Yang, Vera Schmitt, Sebastian M\"oller, Simon Ostermann · 18 mai 2026

  LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g., a 1-5 rating vs. a True/False label). Existing diagnoses of these format-induced inconsistencies stop at the input-outp…

• Margin-Adaptive Confidence Ranking for Reliable LLM Judgement

  Gaojie Jin, Yong Tao, Lijia Yu, Tianjin Huang · 18 mai 2026

  Jung et al. (2025) introduce a hypothesis testing framework for guaranteeing agreement between large language models (LLMs) and human judgments, relying on the assumption that the model's estimated confidence is monotonic with respect to human-disagreement risk. In practice, however, this assumption…

• Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

  Yihang Chen, Pin Qian, Su Wang, Sipeng Zhang, Huan Xu, Shuhuai Lin, Xinpeng Wei · 18 mai 2026

  The Context-Compliance Regime in Retrieval-Augmented Generation (RAG) occurs when retrieved context dominates the final answer even when it conflicts with the model's parametric knowledge. Accuracy alone does not reveal how retrieved context causally shapes answers under such conflict. We introduce …

• AGOP-IxG: A Gradient Covariance Filter for Local Feature Attribution on Tabular Data, with a Controlled Benchmark

  Raj Kiran Gupta Katakam · 18 mai 2026

  Automated machine learning pipelines increasingly produce models whose predictions must be explained to end users, auditors, and downstream decision systems. The most widely used feature attribution methods (SHAP, Integrated Gradients, LIME) are typically chosen by convention rather than measured fi…

• Process Rewards with Learned Reliability

  Jinyuan Li, Langlin Huang, Chengsong Huang, Shaoyang Xu, Donghong Cai, Yuyi Yang, Wenxuan Zhang, Jiaxin Huang · 18 mai 2026

  Process Reward Models (PRMs) provide step-level feedback for reasoning, but current PRMs usually output only a single reward score for each step. Downstream methods must therefore treat imperfect step-level reward predictions as reliable decision signals, with no indication of when these predictions…

• Deterministic Event-Graph Substrates as World Models for Counterfactual Reasoning

  Fabio Rovai · 18 mai 2026

  We study event-graph substrates: a class of world models that represent agent state as an append-only log of typed RDF triples and answer counterfactual queries by forking the log under a structured intervention vocabulary. Substrates are inspectable at the triple level, support exact counterfactual…