Explainable Artificial Intelligence (XAI)

Derniers papiers

• Distribution-informed Efficient Conformal Prediction for Full Ranking

  Wenbo Liao, Huipeng Huang, Chen Jia, Huajun Xi, Hao Zeng, Hongxin Wei · 2 février 2026

  Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications. Recent work offers a rigorous solution using conformal prediction in a full ranking scenario, which aims to construct prediction sets for the absolute ranks of test items based on the relative r…

• Symmetry Breaking in Transformers for Efficient and Interpretable Training

  Eva Silverstein, Daniel Kunin, Vasudev Shyam · 2 février 2026

  The attention mechanism in its standard implementation contains extraneous rotational degrees of freedom that are carried through computation but do not affect model activations or outputs. We introduce a simple symmetry-breaking protocol that inserts a preferred direction into this rotational space…

• An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations

  Seonghwan Park, Jueun Mun, Donghyun Oh, Namhoon Lee · 2 février 2026

  Concept bottleneck models (CBMs) ensure interpretability by decomposing predictions into human interpretable concepts. Yet the annotations used for training CBMs that enable this transparency are often noisy, and the impact of such corruption is not well understood. In this study, we present the fir…

• Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Features

  Yiting Liu, Zhi-Hong Deng · 2 février 2026

  Sparse autoencoders (SAEs) have emerged as a powerful technique for decomposing language model representations into interpretable features. Current interpretation methods infer feature semantics from activation patterns, but overlook that features are trained to reconstruct activations that serve co…

• GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations

  Julia Herbinger, Gabriel Laberge, Maximilian Muschalik, Yann Pequignot, Marvin N. Wright, Fabian Fumagalli · 2 février 2026

  Feature-based explanation methods aim to quantify how features influence the model's behavior, either locally or globally, but different methods often disagree, producing conflicting explanations. This disagreement arises primarily from two sources: how feature interactions are handled and how featu…

• Anytime Safe PAC Efficient Reasoning

  Chengyao Yu, Hao Zeng, Youxin Zhu, Jianguo Huang, Huajun Zeng, Bingyi Jing · 2 février 2026

  Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex tasks but suffer from high computational costs and latency. While selective thinking strategies improve efficiency by routing easy queries to non-thinking models, existing approaches often incur uncontrollable errors, …

• Language Model Circuits Are Sparse in the Neuron Basis

  Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann · 2 février 2026

  The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986). Language model interpretability research has thus turned to techniques such as \textit{sparse autoencoders} (SAEs) to decompose the neuron basis into more interpreta…

• ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations

  Joao Fonseca, Julia Stoyanovich · 2 février 2026

  Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. …

• IRPM: Intergroup Relative Preference Modeling for Pointwise Generative Reward Models

  Haonan Song, Qingchen Xie, Huan Zhu, Feng Xiao, Luxi Xing, Liu Kang, Fuzhen Li, Zhiyong Zheng, Feng Jiang, Ziheng Li, Kun Yan, Qingyi Si, Yanghua Xiao, Hongcheng Guo, Fan Yang · 2 février 2026

  Generative Reward Models (GRMs) have demonstrated strong performance in reward modeling, due to their interpretability and potential for refinement through reinforcement learning (RL). However, widely used pairwise GRMs create a computational bottleneck in reinforcement learning from human feedback …

• ReNCE: Learning to Reason by Noise Contrastive Estimation

  Wenzheng Zhang, Karl Stratos · 2 février 2026

  GRPO is a standard approach to endowing pretrained LLMs with reasoning capabilities. It estimates the advantage of an outcome from a group of $K$ outcomes, and promotes those with positive advantages inside a trust region. Since GRPO discriminates between good and bad outcomes softly, it benefits fr…

• Learn from A Rationalist: Distilling Intermediate Interpretable Rationales

  Jiayi Dai, Randy Goebel · 2 février 2026

  Because of the pervasive use of deep neural networks (DNNs), especially in high-stakes domains, the interpretability of DNNs has received increased attention. The general idea of rationale extraction (RE) is to provide an interpretable-by-design framework for DNNs via a select-predict architecture w…

• Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks

  Nathaniel Mitrani Hadida, Sassan Bhanji, Cameron Tice, Puria Radmard · 2 février 2026

  Chain-of-thought (CoT) reasoning provides a significant performance uplift to LLMs by enabling planning, exploration, and deliberation of their actions. CoT is also a powerful tool for monitoring the behaviours of these agents: when faithful, they offer interpretations of the model's decision making…

• Score-based Integrated Gradient for Root Cause Explanations of Outliers

  Phuoc Nguyen, Truyen Tran, Sunil Gupta, Svetha Venkatesh · 2 février 2026

  Identifying the root causes of outliers is a fundamental problem in causal inference and anomaly detection. Traditional approaches based on heuristics or counterfactual reasoning often struggle under uncertainty and high-dimensional dependencies. We introduce SIREN, a novel and scalable method that …

• LLMs Explain't: A Post-Mortem on Semantic Interpretability in Transformer Models

  Alhassan Abdelhalim, Janick Edinger, S\"oren Laue, Michaela Regneri · 2 février 2026

  Large Language Models (LLMs) are becoming increasingly popular in pervasive computing due to their versatility and strong performance. However, despite their ubiquitous use, the exact mechanisms underlying their outstanding performance remain unclear. Different methods for LLM explainability exist, …

• Real-Time Aligned Reward Model beyond Semantics

  Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, Xuefeng Xiao, Hongyan Xie, Li Huaqiu, Songshi Liang, Zhongxiang Dai, Fuzhen Zhuang, Jianxin Li, Yikun Ban, Deqing Wang · 2 février 2026

  Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique for aligning large language models (LLMs) with human preferences, yet it is susceptible to reward overoptimization, in which policy models overfit to the reward model, exploit spurious reward patterns instead of faithfully capt…

• Metric Hub: A metric library and practical selection workflow for use-case-driven data quality assessment in medical AI

  Katinka Becker, Maximilian P. Oppelt, Tobias S. Zech, Martin Seyferth, Sandie Cabon, Vanja Miskovic, Ivan Cimrak, Michal Kozubek, Giuseppe D'Avenio, Ilaria Campioni, Jana Fehr, Kanjar De, Ismail Mahmoudi, Emilio Dolgener Cantu, Laurenz Ottmann, Andreas Kla{\ss}, Galaad Altares, Jackie Ma, Alireza Salehi M., Nadine R. Lang-Richter, Tobias Schaeffter, Daniel Schwabe · 2 février 2026

  Machine learning (ML) in medicine has transitioned from research to concrete applications aimed at supporting several medical purposes like therapy selection, monitoring and treatment. Acceptance and effective adoption by clinicians and patients, as well as regulatory approval, require evidence of t…

• On the Provable Performance Guarantee of Efficient Reasoning Models

  Hao Zeng, Jianguo Huang, Bingyi Jing, Hongxin Wei, Bo An · 2 février 2026

  Large reasoning models (LRMs) have achieved remarkable progress in complex problem-solving tasks. Despite this success, LRMs typically suffer from high computational costs during deployment, highlighting a need for efficient inference. A practical direction of efficiency improvement is to switch the…

• Human-Centered Explainability in AI-Enhanced UI Security Interfaces: Designing Trustworthy Copilots for Cybersecurity Analysts

  Mona Rajhans · 2 février 2026

  Artificial intelligence (AI) copilots are increasingly integrated into enterprise cybersecurity platforms to assist analysts in threat detection, triage, and remediation. However, the effectiveness of these systems depends not only on the accuracy of underlying models but also on the degree to which…

• XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensics

  Shahid Alam, Zeynep Altiparmak · 2 février 2026

  With the rise of complex cyber devices Cyber Forensics (CF) is facing many new challenges. For example, there are dozens of systems running on smartphones, each with more than millions of downloadable applications. Sifting through this large amount of data and making sense requires new techniques, s…

• SAFER: Probing Safety in Reward Models with Sparse Autoencoder

  Wei Shi, Ziyuan Xie, Sihang Li, Xiang Wang · 2 février 2026

  Reinforcement learning from human feedback (RLHF) is a key paradigm for aligning large language models (LLMs) with human values, yet the reward models at its core remain largely opaque. In this work, we present Sparse Autoencoder For Enhanced Reward model (\textbf{SAFER}), a novel framework for inte…

• Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings

  Safal Shrestha, Minwu Kim, Aadim Nepal, Anubhav Shrestha, Keith Ross · 2 février 2026

  Designing effective reasoning-capable LLMs typically requires training using Reinforcement Learning with Verifiable Rewards (RLVR) or distillation with carefully curated Long Chain of Thoughts (CoT), both of which depend heavily on extensive training data. This creates a major challenge when the amo…

• Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success

  Luca Zhou, Bo Zhao, Rose Yu, Emanuele Rodol\`a · 2 février 2026

  Model merging combines knowledge from separately fine-tuned models, yet success factors remain poorly understood. While recent work treats mergeability as an intrinsic property, we show with an architecture-agnostic framework that it fundamentally depends on both the merging method and the partner t…

• EntroCut: Entropy-Guided Adaptive Truncation for Efficient Chain-of-Thought Reasoning in Small-scale Large Reasoning Models

  Hongxi Yan, Qingjie Liu, Yunhong Wang · 2 février 2026

  Large Reasoning Models (LRMs) excel at complex reasoning tasks through extended chain-of-thought generation, but their reliance on lengthy intermediate steps incurs substantial computational cost. We find that the entropy of the model's output distribution in early reasoning steps reliably distingui…

• RAudit: A Blind Auditing Protocol for Large Language Model Reasoning

  Edward Y. Chang, Longling Geng · 2 février 2026

  Inference-time scaling can amplify reasoning pathologies: sycophancy, rung collapse, and premature certainty. We present RAudit, a diagnostic protocol for auditing LLM reasoning without ground truth access. The key constraint is blindness: the auditor evaluates only whether derivation steps support …

• Uncertainty-Aware Extrapolation in Bayesian Oblique Trees

  Viktor Andonovikj, Sa\v{s}o D\v{z}eroski, Pavle Bo\v{s}koski · 2 février 2026

  Decision trees are widely used due to their interpretability and efficiency, but they struggle in regression tasks that require reliable extrapolation and well-calibrated uncertainty. Piecewise-constant leaf predictions are bounded by the training targets and often become overconfident under distrib…