Explainable Artificial Intelligence (XAI)

Derniers papiers

• LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs

  Chacha Chen, Matthew J\"orke, Adam Goli\'nski, Masha Fedzechkina, Guillermo Sapiro, Sinead Williamson, Nicholas Foti · 28 mai 2026

  Modern AI systems are being deployed in complex domains such as medicine, science, and law, where it is important that they not only produce correct answers, but also represent and update uncertain beliefs about the world as new evidence arrives. We introduce the novel technique of studying LLMs as …

• Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness

  Jaechang Kim, Sunung Mun, Seungjoon Lee, Jaewoong Cho, Jungseul Ok · 28 mai 2026

  Explainable AI (XAI) helps users interpret model behavior and identify potential faults. Agentic XAI systems use Large Language Models (LLMs) to make explanations more accessible through natural-language interaction, but they can also produce plausible yet unfaithful explanations. This risk arises b…

• CAREF: Calibration-Aware Regularization for Explanation Faithfulness Without Rationale Supervision

  Naphat Nithisopa, Teerapong Panboonyuen · 28 mai 2026

  We introduce CAREF, a parameter-efficient fine-tuning framework that jointly optimizes predictive accuracy and explanation faithfulness via calibration-aware regularization. At its core, CAREF couples entropy-based calibration with token-level sparsity control through a single unified loss, the Cali…

• Local MDI+: Local Feature Importances for Tree-Based Models

  Zhongyuan Liang, Zachary T. Rewolinski, Abhineet Agarwal, Tiffany M. Tang, Bin Yu · 28 mai 2026

  Tree-based ensembles such as random forests remain the go-to for tabular data over deep learning models due to their prediction performance and computational efficiency. These advantages have led to their widespread deployment in high-stakes domains, where interpretability is essential for ensuring …

• Let the Results Speak: A Replication-First Paradigm for LLM Behavioral Benchmarking

  Yuming (Rapheal), Huang, Yao Liu, Lei Wang, Junchen Wan · 28 mai 2026

  Subjective evaluation of LLM behavior -- empathy, restraint, calibrated emotional tone -- is hard. Human inter-rater agreement on such qualities saturates near rho ~ 0.45, and an LLM-as-judge proxy alone risks circularity: a judge sharing the target's training cohort cannot independently verify it. …

• Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures

  Yongjin Cui, Xiaohui Fan, Huajun Chen · 28 mai 2026

  Transformer has significantly propelled the development of artificial intelligence, and certainly the development of agents as well. We categorize attention structures of Transformer into two types based on the source of the input information: homogenous and heterogenous attention structures. Hetero…

• Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors

  Luyang Fang, Yongkai Chen, Jiazhang Cai, Ping Ma, Wenxuan Zhong · 28 mai 2026

  Knowledge distillation is a powerful method for model compression, enabling the efficient deployment of complex deep learning models (teachers), including large language models. However, its underlying statistical mechanisms remain unclear, and uncertainty evaluation is often overlooked, especially …

• DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis

  Qiaohong Zhang, Weihao Ye, Jialong Chen, Yi Luo, BoYuan Li, Bowen Deng, Zibin Zheng, Jianhao Lin, Wei-Shi Zheng, Chuan Chen · 28 mai 2026

  Autonomous data analysis agents are increasingly expected to conduct exploratory analysis with limited human guidance about data. However, existing benchmarks typically evaluate such agents in prior-guided settings, providing selected data sources, explicit data schemas, or cleaned data, thereby und…

• Revisiting Metafeatures to Explain Model Differences on Tabular Data

  Markus Herre, Andrej Tschalzev, Sascha Marton, Christian Bartelt · 28 mai 2026

  With the rise of tabular foundation models alongside traditional models still performing well on many tasks, choosing the right model for a tabular dataset remains difficult. We investigate whether dataset meta-features can explain performance gaps between model families on tabular prediction tasks.…

• Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions

  Thomas Vitry, Kieran Edgeworth, Stefan Wermter, Jae Hee Lee · 28 mai 2026

  Vision classifiers can exploit spurious correlations, achieving high in-distribution accuracy yet failing under distribution shift. Existing approaches to bias mitigation and analysis often depend on curated datasets, spurious-attribute or group labels, or retraining, which may be infeasible once a …

• Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information

  Renjie Gu, Jiaxu Li, Yihao Wang, Yun Yue, Hansong Xiao, Yefei Chen, Yuan Wang, Chunxiao Guo, Pei Wei, Jinjie Gu, Yixin Cao · 28 mai 2026

  We highlight a failure mode of large reasoning models on questions with insufficient information: models may recognize that a problem is under-specified, yet still continue reasoning and produce unsupported final answers instead of abstaining. We formalize this mismatch as the detection-to-abstentio…

• Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure

  Max Lamparth, Daniel Fein, Andreas Haupt, Marcel Hussing, Mykel J. Kochenderfer · 28 mai 2026

  Single-axis mitigations of reward-model biases (e.g., reducing proxy reliance on length, sycophancy, or style) can rotate optimization pressure onto correlated proxies rather than eliminate it, a failure mode we call reward bias substitution. The failure is enabled by a measurement-versus-optimizati…

• Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

  Yaoyang Luo, Zhi Zheng, Ziwei Zhao, Tong Xu, Zhao Jielun, Wenjun Xue, Yong Chen, Enhong Chen · 28 mai 2026

  Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. However, malicious agents in MAS may inject misinformation to mislead other agents and disrupt system performance, givi…

• CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

  Linas Nasvytis, Simon Jerome Han, Ben Prystawski, Satchel Grant, Noah D. Goodman, Judith E. Fan · 28 mai 2026

  Language models can use verifiable rewards to improve at a wide variety of reasoning tasks. However, both parametric (e.g. RLVR) and non-parametric (e.g. prompt optimization) approaches to doing so typically require hundreds of training samples and thousands of model rollouts, making them expensive …

• Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data

  Tom\'as Pereira, Jo\~ao Vitorino, Eva Maia, Isabel Pra\c{c}a · 28 mai 2026

  Despite the wide use of explainability techniques to attempt to understand the behavior of Artificial Intelligence (AI), the generated explanations may not always be reliable. An explanation can appear plausible to humans but fail to capture the internal reasoning of a model, particularly when deali…

• Auditable Decision Models with Learned Abstention and Real-Time Steering

  Sankaranarayanan Palamadai Chandrasekaran · 28 mai 2026

  Production AI systems often operate with incomplete, conflicting, or insufficient evidence. Forced classifiers collapse such cases into action labels, while generative systems can produce outputs that are difficult to interpret as auditable execution decisions. We study operational decision control …

• Integrated and Cross-Architecture Interpretation of LLM Reasoning

  Leonardo Matthew Yauw, Wei-Bin Kou, Yujiu Yang · 28 mai 2026

  Understanding how LLMs reason is hindered by a practical asymmetry: while their generated outputs are observable, the underlying reasoning patterns remain opaque. Relying on single probes, such as Mutual Information Peak (MIP) or Deep-Thinking Ratio (DTR), risks underestimating the genuine inferenti…

• Semantic Optimal Transport for Sparse Autoencoder Feature Matching and Circuit Compression

  Tue M. Cao, Nguyen Do, My T. Thai · 28 mai 2026

  Sparse autoencoders (SAEs) have become a central tool for interpreting language models. However, two key SAE analyses that remain difficult to scale are (1) matching semantically similar features across multi-layers and (2) compressing large feature circuits into interpretable supernodes. Although t…

• PINE: Pruning Boosted Tree Ensembles with Conformal In-Distribution Prediction Equivalence

  Haruki Yajima, Yusuke Matsui · 28 mai 2026

  Tree ensembles are machine learning models with strong predictive performance and interpretability, and remain widely used for tabular data. Standard pruning methods for tree ensembles typically optimize an accuracy-compression trade-off and may change a subset of predictions, potentially compromisi…

• REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading

  Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu · 28 mai 2026

  Open-ended grading is central to equitable and personalized education, yet manual grading remains time-consuming and costly, underscoring the need for automated grading systems. Although recent neural and large language model (LLM) based systems have demonstrated superior performance, they are typic…

• High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models

  Jos\'e Lucas De Melo Costa, Fabrice Popineau, Arpad Rimmel, Bich-Li\^en Doan · 28 mai 2026

  Recent Tabular Foundation Models (TFMs) have demonstrated state-of-the-art predictive performance, often surpassing Gradient-Boosted Decision Trees (GBDTs). However, the trustworthiness of these models, particularly their uncertainty quantification, has been largely overlooked. We investigate this g…

• When Interpretability Is Unequally Distributed: Fairness in Hybrid Interpretable Models

  Ziba Jabbar Zare, Ulrich A\"ivodji, Julien Ferry, Thibaut Vidal · 28 mai 2026

  Hybrid interpretable models combine a transparent component with a black-box model by assigning some examples to the former and deferring the rest to the latter. While this design enables flexible tradeoffs between accuracy and interpretability, it also raises a distinct procedural fairness concern:…

• Tell Me a Story! Narrative-Driven XAI with Large Language Models

  David Martens, James Hinns, Camille Dams, Mark Vergouwen, Theodoros Evgeniou · 28 mai 2026

  In many AI applications today, the predominance of black-box machine learning models, due to their typically higher accuracy, amplifies the need for Explainable AI (XAI). Existing XAI approaches, such as the widely used SHAP values or counterfactual (CF) explanations, are arguably often too technica…

• Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning

  Zihan Liang, Yufei Ma, Ben Chen, Zhipeng Qian, Xuxin Zhang, Huangyu Dai, Lingtao Mao · 27 mai 2026

  Post-training has become the dominant recipe for turning a language model into a competent search-augmented reasoning agent. A line of recent work pushes its performance further by adding elaborate machinery on top of this standard pipeline. These augmentations import external supervision from stron…

• MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability

  Barsat Khadka · 27 mai 2026

  Mechanistic interpretability has identified small sets of attention heads that implement specific behaviours in transformer language models, but recovering these circuits typically requires a bespoke analytical pipeline for each new task. We recast circuit discovery as a reinforcement-learning probl…