Explainable Artificial Intelligence (XAI)

Derniers papiers

• Opinion: Learning Intuitive Physics May Require More than Visual Data

  Ellen Su, Solim Legris, Todd M. Gureckis, Mengye Ren · 9 décembre 2025

  Humans expertly navigate the world by building rich internal models founded on an intuitive understanding of physics. Meanwhile, despite training on vast quantities of internet video data, state-of-the-art deep learning models still fall short of human-level performance on intuitive physics benchmar…

• Collaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support

  Raunak Jain, Mudita Khurana · 9 décembre 2025

  LLM-based agents are rapidly being plugged into expert decision-support, yet in messy, high-stakes settings they rarely make the team smarter: human-AI teams often underperform the best individual, experts oscillate between verification loops and over-reliance, and the promised complementarity does …

• Depth-Wise Activation Steering for Honest Language Models

  Gracjan G\'oral, Marysia Winkels, Steven Basart · 9 décembre 2025

  Large language models sometimes assert falsehoods despite internally representing the correct answer, failures of honesty rather than accuracy, which undermines auditability and safety. Existing approaches largely optimize factual correctness or depend on retraining and brittle single-layer edits, o…

• One Sample is Enough to Make Conformal Prediction Robust

  Soroush H. Zargarbashi, Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski · 9 décembre 2025

  For any black-box model, conformal prediction (CP) returns prediction sets guaranteed to include the true label with high adjustable probability. Robust CP (RCP) extends the guarantee to the worst case noise up to a pre-defined magnitude. For RCP, a well-established approach is to use randomized smo…

• KAN-Dreamer: Benchmarking Kolmogorov-Arnold Networks as Function Approximators in World Models

  Chenwei Shi, Xueyu Luan · 9 décembre 2025

  DreamerV3 is a state-of-the-art online model-based reinforcement learning (MBRL) algorithm known for remarkable sample efficiency. Concurrently, Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to Multi-Layer Perceptrons (MLPs), offering superior parameter efficiency and int…

• Clinical Interpretability of Deep Learning Segmentation Through Shapley-Derived Agreement and Uncertainty Metrics

  Tianyi Ren, Daniel Low, Pittra Jaengprajak, Juampablo Heras Rivera, Jacob Ruzevick, Mehmet Kurt · 9 décembre 2025

  Segmentation is the identification of anatomical regions of interest, such as organs, tissue, and lesions, serving as a fundamental task in computer-aided diagnosis in medical imaging. Although deep learning models have achieved remarkable performance in medical image segmentation, the need for expl…

• LLMs Judging LLMs: A Simplex Perspective

  Patrick Vossler, Fan Xia, Yifan Mai, Adarsh Subbaswamy, Jean Feng · 9 décembre 2025

  Given the challenge of automatically evaluating free-form outputs from large language models (LLMs), an increasingly common solution is to use LLMs themselves as the judging mechanism, without any gold-standard scores. Implicitly, this practice accounts for only sampling variability (aleatoric uncer…

• A Geometric Unification of Concept Learning with Concept Cones

  Alexandre Rocchi--Henry, Thomas Fel, Gianni Franchi · 9 décembre 2025

  Two traditions of interpretability have evolved side by side but seldom spoken to each other: Concept Bottleneck Models (CBMs), which prescribe what a concept should be, and Sparse Autoencoders (SAEs), which discover what concepts emerge. While CBMs use supervision to align activations with human-la…

• $\phi$-test: Global Feature Selection and Inference for Shapley Additive Explanations

  Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool, Gisung Oh · 9 décembre 2025

  We propose $\phi$-test, a global feature-selection and significance procedure for black-box predictors that combines Shapley attributions with selective inference. Given a trained model and an evaluation dataset, $\phi$-test performs SHAP-guided screening and fits a linear surrogate on the screened …

• Going All-In on LLM Accuracy: Fake Prediction Markets, Real Confidence Signals

  Michael Todasco (Visiting Fellow at the James Silberrad Center for Artificial Intelligence, San Diego State University) · 9 décembre 2025

  Large language models are increasingly used to evaluate other models, yet these judgments typically lack any representation of confidence. This pilot study tests whether framing an evaluation task as a betting game (a fictional prediction market with its own LLM currency) improves forecasting accura…

• Countering Overfitting with Counterfactual Examples

  Flavio Giorgi, Fabiano Veglianti, Fabrizio Silvestri, Gabriele Tolomei · 9 décembre 2025

  Overfitting is a well-known issue in machine learning that occurs when a model struggles to generalize its predictions to new, unseen data beyond the scope of its training set. Traditional techniques to mitigate overfitting include early stopping, data augmentation, and regularization. In this work,…

• Always Keep Your Promises: DynamicLRP, A Model-Agnostic Solution To Layer-Wise Relevance Propagation

  Kevin Lee, Pablo Millan Arias · 9 décembre 2025

  Layer-wise Relevance Propagation (LRP) provides principled attribution for neural networks through conservation properties and foundations in Deep Taylor Decomposition. However, existing implementations operate at the module level, requiring architecture-specific propagation rules and modifications.…

• Parent-Guided Semantic Reward Model (PGSRM): Embedding-Based Reward Functions for Reinforcement Learning of Transformer Language Models

  Alexandr Plashchinsky · 9 décembre 2025

  We introduce the Parent-Guided Semantic Reward Model (PGSRM), a lightweight reward framework for reinforcement learning (RL) of transformer language models. PGSRM replaces binary correctness signals, human preference data, and trained reward models with a simple signal: cosine similarity between a p…

• Know your Trajectory -- Trustworthy Reinforcement Learning deployment through Importance-Based Trajectory Analysis

  Clifford F, Devika Jay, Abhishek Sarkar, Satheesh K Perepu, Santhosh G S, Kaushik Dey, Balaraman Ravindran · 9 décembre 2025

  As Reinforcement Learning (RL) agents are increasingly deployed in real-world applications, ensuring their behavior is transparent and trustworthy is paramount. A key component of trust is explainability, yet much of the work in Explainable RL (XRL) focuses on local, single-step decisions. This pape…

• ContextualSHAP : Enhancing SHAP Explanations Through Contextual Language Generation

  Latifa Dwiyanti, Sergio Ryan Wibisono, Hidetaka Nambo · 9 décembre 2025

  Explainable Artificial Intelligence (XAI) has become an increasingly important area of research, particularly as machine learning models are deployed in high-stakes domains. Among various XAI approaches, SHAP (SHapley Additive exPlanations) has gained prominence due to its ability to provide both gl…

• On the Theoretical Foundation of Sparse Dictionary Learning in Mechanistic Interpretability

  Yiming Tang, Harshvardhan Saini, Yizhen Liao, Dianbo Liu · 8 décembre 2025

  As AI models achieve remarkable capabilities across diverse domains, understanding what representations they learn and how they process information has become increasingly important for both scientific progress and trustworthy deployment. Recent works in mechanistic interpretability have shown that …

• Measuring the Effect of Background on Classification and Feature Importance in Deep Learning for AV Perception

  Anne Sielemann, Valentin Barner, Stefan Wolf, Masoud Roschani, Jens Ziehn, Juergen Beyerer · 8 décembre 2025

  Common approaches to explainable AI (XAI) for deep learning focus on analyzing the importance of input features on the classification task in a given model: saliency methods like SHAP and GradCAM are used to measure the impact of spatial regions of the input image on the classification result. Combi…

• Rolling in the deep of cognitive and AI biases

  Athena Vakali, Nicoleta Tantalaki · 8 décembre 2025

  Nowadays, we delegate many of our decisions to Artificial Intelligence (AI) that acts either in solo or as a human companion in decisions made to support several sensitive domains, like healthcare, financial services and law enforcement. AI systems, even carefully designed to be fair, are heavily cr…

• Variational Uncertainty Decomposition for In-Context Learning

  I. Shavindra Jayasekera, Jacob Si, Filippo Valdettaro, Wenlong Chen, A. Aldo Faisal, Yingzhen Li · 8 décembre 2025

  As large language models (LLMs) gain popularity in conducting prediction tasks in-context, understanding the sources of uncertainty in in-context learning becomes essential to ensuring reliability. The recent hypothesis of in-context learning performing predictive Bayesian inference opens the avenue…

• Improving Local Fidelity Through Sampling and Modeling Nonlinearity

  Sanjeev Shrestha, Rahul Dubey, Hui Liu · 8 décembre 2025

  With the increasing complexity of black-box machine learning models and their adoption in high-stakes areas, it is critical to provide explanations for their predictions. Local Interpretable Model-agnostic Explanation (LIME) is a widely used technique that explains the prediction of any classifier b…

• Sparse Attention Post-Training for Mechanistic Interpretability

  Florent Draye, Anson Lei, Ingmar Posner, Bernhard Sch\"olkopf · 8 décembre 2025

  We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-loss objective, we show on models up to 1B parameters that it is possible to retain the original pretraining loss while …

• PCS Workflow for Veridical Data Science in the Age of AI

  Zachary T. Rewolinski, Bin Yu · 4 décembre 2025

  Data science is a pillar of artificial intelligence (AI), which is transforming nearly every domain of human activity, from the social and physical sciences to engineering and medicine. While data-driven findings in AI offer unprecedented power to extract insights and guide decision-making, many are…

• ZIP-RC: Optimizing Test-Time Compute via Zero-Overhead Joint Reward-Cost Prediction

  Rohin Manvi, Joey Hong, Tim Seyde, Maxime Labonne, Mathias Lechner, Sergey Levine · 4 décembre 2025

  Large language models excel at reasoning but lack key aspects of introspection, including anticipating their own success and the computation required to achieve it. Humans use real-time introspection to decide how much effort to invest, when to make multiple attempts, when to stop, and when to signa…

• Deep Unfolding: Recent Developments, Theory, and Design Guidelines

  Nir Shlezinger, Santiago Segarra, Yi Zhang, Dvir Avrahami, Zohar Davidov, Tirza Routtenberg, Yonina C. Eldar · 4 décembre 2025

  Optimization methods play a central role in signal processing, serving as the mathematical foundation for inference, estimation, and control. While classical iterative optimization algorithms provide interpretability and theoretical guarantees, they often rely on surrogate objectives, require carefu…

• Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretability

  Tuomas Oikarinen, Ge Yan, Akshay Kulkarni, Tsui-Wei Weng · 4 décembre 2025

  Interpreting individual neurons or directions in activation space is an important topic in mechanistic interpretability. Numerous automated interpretability methods have been proposed to generate such explanations, but it remains unclear how reliable these explanations are, and which methods produce…