Explainable Artificial Intelligence (XAI)

Latest papers

• Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States

  Yunho Choi, Jongwon Lim, Woojin Ahn, Minjae Oh, Jeonghoon Shim, Yohan Jo · 11 May 2026

  Reinforcement learning with verifiable rewards (RLVR) for Large Reasoning Models hinges on baseline estimation for variance reduction, but existing approaches pay a heavy price: PPO requires a policy-model scale critic, while GRPO needs multiple rollouts per prompt to keep its empirical group mean s…

• Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation

  Sayantan Dasgupta, Trevor Cohn, Timothy Baldwin · 11 May 2026

  The core learning signal used in language model distillation is the standard Kullback-Leibler (KL) divergence between the student and teacher distributions. Traditional KL divergence tends to be dominated by the next tokens with the highest probabilities, i.e., the teacher's modes, thereby diminishi…

• Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States

  Yunho Choi, Jongwon Lim, Woojin Ahn, Minjae Oh, Jeonghoon Shim, Yohan Jo · 11 May 2026

  Reinforcement learning with verifiable rewards (RLVR) for Large Reasoning Models hinges on baseline estimation for variance reduction, but existing approaches pay a heavy price: PPO requires a policy-model scale critic, while GRPO needs multiple rollouts per prompt to keep its empirical group mean s…

• ProtoSSL: Interpretable Prototype Learning from Unlabeled Time-Series Data

  Steven Song, Sahil Sethi, Brett Beaulieu-Jones, Robert L. Grossman · 11 May 2026

  In time-series domains where both predictive performance and interpretability are essential, deep neural networks achieve strong results but provide limited insight into how their predictions are made. Projection-based prototype networks address this limitation by grounding predictions in similarity…

• A Differentiable Bayesian Relaxation for Latent Partial-Order Inference

  Dongqing Li, Geoff K. Nicholls, Shiyi Sun, You Luo · 11 May 2026

  Many ranking and agent trace datasets are recorded as linear orders even though their latent structure is only partially ordered. This is especially common in agent and workflow traces, where observed order may reflect arbitrary linearization rather than true prerequisites. We introduce a differenti…

• The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts

  Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero, Richard Baraniuk · 11 May 2026

  The Linear Representation Hypothesis (LRH) identifies features of a trained deep network (DN) as linear directions in the activation spaces, i.e., output spaces of intermediate layers. This characterization decouples the input-output maps learned by a DN from the organization of feature directions i…

• Confidence-Aware Alignment Makes Reasoning LLMs More Reliable

  Kejia Chen, Jiawen Zhang, Yihong Wu, Kewei Gao, Jian Lou, Zunlei Feng, Mingli Song, Ruoxi Jia · 11 May 2026

  Large reasoning models often reach correct answers through flawed intermediate steps, creating a gap between final accuracy and reasoning reliability. Existing alignment strategies address this with external verifiers or massive sampling, limiting scalability. In this work, we introduce CASPO (Confi…

• Why Self-Inconsistency Arises in GNN Explanations and How to Exploit It

  Wenxin Tai, Yaqian Liu, Ting Zhong, Fan Zhou · 11 May 2026

  Recent work has observed that explanations produced by Self-Interpretable Graph Neural Networks (SI-GNNs) can be self-inconsistent: when the model is reapplied to its own explanatory graph subset, it may produce a different explanation. However, why self-inconsistency arises remains poorly understoo…

• Approximation-Free Differentiable Oblique Decision Trees

  Subrat Prasad Panda, Blaise Genest, Arvind Easwaran · 11 May 2026

  Decision Trees (DTs) are widely used in safety-critical domains such as medical diagnosis, valued for their interpretability and effectiveness on tabular data. However, training accurate oblique DTs is challenging due to complex optimization landscapes and overfitting risks, particularly in regressi…

• LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs

  Chacha Chen, Matthew J\"orke, Adam Goli\'nski, Masha Fedzechkina, Guillermo Sapiro, Sinead Williamson, Nicholas Foti · 11 May 2026

  Modern AI systems are being deployed in complex domains such as medicine, science, and law, where it is important that they not only produce correct answers, but also represent and update uncertain beliefs about the world as new evidence arrives. We introduce the novel technique of studying LLMs as …

• Interpreting Reinforcement Learning Agents with Susceptibilities

  Chris Elliott, Einar Urdshals, David Quarel, Daniel Murfet · 11 May 2026

  Susceptibilities are a technique for neural network interpretability that studies the response of posterior expectation values of observables to perturbations of the loss. We generalize this construction to the setting of the regret in deep reinforcement learning and investigate the utility of susce…

• Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

  Roger Creus Castanyer, Pablo Samuel Castro, Glen Berseth · 11 May 2026

  AI agent research spans a wide spectrum: from RL agents that learn from scratch to foundation model agents that leverage pre-trained knowledge, yet no unified benchmark enables fair comparison across these approaches. We present Agentick, a benchmark for sequential decision-making agents designed to…

• PLOT: Progressive Localization via Optimal Transport in Neural Causal Abstraction

  Jonathn Chang, Arya Datla, Ziv Goldfeld · 11 May 2026

  Causal abstraction offers a principled framework for mechanistic interpretability, aligning a high-level causal model with the low-level computation realized by a neural network through counterfactual intervention analysis. Existing methods such as distributed alignment search (DAS) learn expressive…

• Beyond the Black Box: Interpretability of Agentic AI Tool Use

  Hariom Tatsat, Ariye Shater · 11 May 2026

  AI agents are promising for high-stakes enterprise workflows, but dependable deployment remains limited because tool-use failures are difficult to diagnose and control. Agents may skip required tool calls, invoke tools unnecessarily, or take actions whose consequence becomes visible only after execu…

• Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents

  Deeraj S K, Sadhana Devarajan, Krishna Mehra, Sudhakar Mishra · 11 May 2026

  Reinforcement learning from verifiable emotion rewards RLVER has produced language models with strong empathetic performance, evaluated on benchmarks that assume cooperative, honest users. Yet real emotional interactions systematically violate this assumption: users gaslight, escalate, and pressure …

• Mitigating Cognitive Bias in RLHF by Altering Rationality

  Tiffany Horter, Andrew Markham, Niki Trigoni, Serena Booth · 11 May 2026

  How can we make models robust to even imperfect human feedback? In reinforcement learning from human feedback (RLHF), human preferences over model outputs are used to train a reward model that assigns scalar values to responses. Because these rewards are inferred from pairwise comparisons, this lear…

• Abductive Reasoning with Probabilistic Commonsense

  Joseph Cotnareanu, Chiara Roverato, Han Zhou, Didier Chetelat, Yingxue Zhang, Mark Coates · 11 May 2026

  Recent efforts to improve the reasoning abilities of Large Language Models (LLMs) have focused on integrating formal logic solvers within neurosymbolic frameworks. A key challenge is that formal solvers lack commonsense world knowledge, preventing them from making reasoning steps that humans find ob…

• Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning

  Manish Bhattarai, Ismael Boureima, Nishath Rajiv Ranasinghe, Scott Pakin, Dan O'Malley · 11 May 2026

  We argue that decomposing reward into weighted, verifiable criteria and using an LLM judge to score them provides a partial-credit optimization signal: instead of a binary outcome or a single holistic score, each response is graded along multiple task-specific criteria. We formalize \emph{rubric-gro…

• When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoning

  Vasilis Niarchos, Constantinos Papageorgakis, Alexander G. Stapleton, Sokratis Trifinopoulos · 11 May 2026

  As large language models (LLMs) show increasing promise on research-level physics reasoning tasks and agentic AI becomes more common, a practical question emerges: How does the interaction between researchers and agents affect the results? We study this using SCALAR (Structured Critic--Actor Loop fo…

• MAVEN: Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditing

  Yinsheng Yao, Jiehao Tang, Zhaozhen Yang, Dawei Cheng · 11 May 2026

  While explicit reasoning trajectories enhance model interpretability, existing paradigms often rely on monolithic chains that lack intermediate verification, allowing early errors to cascade unchecked. This lack of modularity impedes granular auditing and compromises the epistemic trust required for…

• Harnessing Pre-Resolution Signals for Future Prediction Agents

  Chuyang Wei, Maohang Gao, Zhixin Han, Kefei Chen, Yu Zhuang, Haoxiang Guan, Yanzhi Zhang, Yilin Cheng, Xiren Zhou, Huanhuan Chen, Jian Li, Jiyan He, Yu Shi, Yitong Duan, Shuxin Zheng · 11 May 2026

  Many high-stakes decisions depend on forecasts made before outcomes are known. In this future prediction setting, the central challenge is that public evidence evolves over time, while the main supervision signal arrives only after resolution: the realized outcome mainly assesses final correctness, …

• Replicating Human Motivated Reasoning Studies with LLMs

  Neeley Pate, Adiba Mahbub Proma, Hangfeng He, James N. Druckman, Daniel C. Molden, Gourab Ghoshal, Ehsan Hoque · 11 May 2026

  Motivated reasoning - the idea that individuals processing information may be motivated to either arrive at accurate beliefs or arrive at desired conclusions - has been well-explored as a human phenomenon. However, it remains unclear whether base LLMs are affected by motivational manipulations. Repl…

• Theoretical Limits of Language Model Alignment

  Lucas Monteiro Paes, Natalie Mackraz, Barry-John Theobald, Federico Danieli · 11 May 2026

  Language model (LM) alignment improves model outputs to reflect human preferences while preserving the capabilities of the base model. The most common alignment approaches are (i) reinforcement learning, which maximizes the expected reward under a KL-divergence constraint, and (ii) best-of-$N$ align…

• Position: Mechanistic Interpretability Must Disclose Identification Assumptions for Causal Claims

  Zezheng Lin, Fengming Liu · 11 May 2026

  Mechanistic interpretability papers increasingly use causal vocabulary: circuits, mediators, causal abstraction, monosemanticity. Such claims require explicit identification assumptions. A purposive audit of 10 papers across four methodological strands finds no dedicated identification-assumptions s…

• Amortized Linear-time Exact Shapley Value for Product-Kernel Methods

  Majid Mohammadi, Siu Lun Chau, Krikamol Muandet · 8 May 2026

  Kernel methods are widely used in machine learning and statistics for their flexibility and expressive power, yet their black-box nature limits adoption in high-stakes applications. Shapley value-based attribution methods such as SHAP, and kernel-specific adaptations including RKHS-SHAP, provide a p…