Adversarial Robustness in Machine Learning

Neueste Paper

• Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions

  Mame Diarra Toure, David A. Stephens · 25. Februar 2026

  In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class. We decompose MI into …

• When Backdoors Go Beyond Triggers: Semantic Drift in Diffusion Models Under Encoder Attacks

  Shenyang Chen, Liuwan Zhu · 25. Februar 2026

  Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this paradigm, demonstrating that encoder-side poisoning induces persistent, trigger-free semantic corruption that fundamentally reshapes the representation m…

• DANCE: Doubly Adaptive Neighborhood Conformal Estimation

  Brandon R. Feng, Brian J. Reich, Daniel Beaglehole, Xihaier Luo, David Keetae Park, Shinjae Yoo, Zhechao Huang, Xueyu Mao, Olcay Boz, Jungeum Kim · 25. Februar 2026

  The recent developments of complex deep learning models have led to unprecedented ability to accurately predict across multiple data representation types. Conformal prediction for uncertainty quantification of these models has risen in popularity, providing adaptive, statistically-valid prediction s…

• Watermarking Degrades Alignment in Language Models: Analysis and Mitigation

  Apurv Verma, NhatHai Phan, Shubhendu Trivedi · 25. Februar 2026

  Watermarking has become a practical tool for tracing language model outputs, but it modifies token probabilities at inference time, which were carefully tuned by alignment training. This creates a tension: how do watermark-induced shifts interact with the procedures intended to make models safe and …

• CREDIT: Certified Ownership Verification of Deep Neural Networks Against Model Extraction Attacks

  Bolin Shen, Zhan Cheng, Neil Zhenqiang Gong, Fan Yao, Yushun Dong · 25. Februar 2026

  Machine Learning as a Service (MLaaS) has emerged as a widely adopted paradigm for providing access to deep neural network (DNN) models, enabling users to conveniently leverage these models through standardized APIs. However, such services are highly vulnerable to Model Extraction Attacks (MEAs), wh…

• Evaluating Proactive Risk Awareness of Large Language Models

  Xuan Luo, Yubin Chen, Zhiyu Hou, Linpu Yu, Geng Tu, Jing Li, Ruifeng Xu · 25. Februar 2026

  As large language models (LLMs) are increasingly embedded in everyday decision-making, their safety responsibilities extend beyond reacting to explicit harmful intent toward anticipating unintended but consequential risks. In this work, we introduce a proactive risk awareness evaluation framework th…

• AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs

  Che Wang, Jiaming Zhang, Ziqi Zhang, Zijie Wang, Yinghui Wang, Jianbo Gao, Tao Wei, Zhong Chen, Wei Yang Bryan Lim · 25. Februar 2026

  The integration of external data services (e.g., Model Context Protocol, MCP) has made large language model-based agents increasingly powerful for complex task execution. However, this advancement introduces critical security vulnerabilities, particularly indirect prompt injection (IPI) attacks. Exi…

• Estimation of Confidence Bounds in Binary Classification using Wilson Score Kernel Density Estimation

  Thorbj{\o}rn Mosekj{\ae}r Iversen, Zebin Duan, Frederik Hagelskj{\ae}r · 25. Februar 2026

  The performance and ease of use of deep learning-based binary classifiers have improved significantly in recent years. This has opened up the potential for automating critical inspection tasks, which have traditionally only been trusted to be done manually. However, the application of binary classif…

• CAGE: A Framework for Culturally Adaptive Red-Teaming Benchmark Generation

  Chaeyun Kim, YongTaek Lim, Kihyun Kim, Junghwan Kim, Minwoo Kim · 25. Februar 2026

  Existing red-teaming benchmarks, when adapted to new languages via direct translation, fail to capture socio-technical vulnerabilities rooted in local culture and law, creating a critical blind spot in LLM safety evaluation. To address this gap, we introduce CAGE (Culturally Adaptive Generation), a …

• No One Size Fits All: QueryBandits for Hallucination Mitigation

  Nicole Cho, William Watson, Alec Koppel, Sumitra Ganesh, Manuela Veloso · 25. Februar 2026

  Advanced reasoning capabilities in Large Language Models (LLMs) have led to more frequent hallucinations; yet most mitigation work focuses on open-source models for post-hoc detection and parameter editing. The dearth of studies focusing on hallucinations in closed-source models is especially concer…

• ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction

  Che Wang, Fuyao Zhang, Jiaming Zhang, Ziqi Zhang, Yinghui Wang, Longtao Huang, Jianbo Gao, Zhong Chen, Wei Yang Bryan Lim · 25. Februar 2026

  Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution. Existing defenses typically rely on strict filtering or refusal mechanisms, which suffer from a critical limitation: over-refu…

• When can we trust untrusted monitoring? A safety case sketch across collusion strategies

  Nelson Gardner-Challis, Jonathan Bostock, Georgiy Kozhevnikov, Morgan Sinclaire, Joan Velja, Alessandro Abate, Charlie Griffin · 25. Februar 2026

  AIs are increasingly being deployed with greater autonomy and capabilities, which increases the risk that a misaligned AI may be able to cause catastrophic harm. Untrusted monitoring -- using one untrusted model to oversee another -- is one approach to reducing risk. Justifying the safety of an untr…

• What Matters For Safety Alignment?

  Xing Li, Hui-Ling Zhen, Lihao Yin, Xianzhi Yu, Zhenhua Dong, Mingxuan Yuan · 25. Februar 2026

  This paper presents a comprehensive empirical study on the safety alignment capabilities. We evaluate what matters for safety alignment in LLMs and LRMs to provide essential insights for developing more secure and reliable AI systems. We systematically investigate and compare the influence of six cr…

• Reliable Abstention under Adversarial Injections: Tight Lower Bounds and New Upper Bounds

  Ezra Edelman, Surbhi Goel · 24. Februar 2026

  We study online learning in the adversarial injection model introduced by [Goel et al. 2017], where a stream of labeled examples is predominantly drawn i.i.d.\ from an unknown distribution $\mathcal{D}$, but may be interspersed with adversarially chosen instances without the learner knowing which ro…

• Advantage-based Temporal Attack in Reinforcement Learning

  Shenghong He · 24. Februar 2026

  Extensive research demonstrates that Deep Reinforcement Learning (DRL) models are susceptible to adversarially constructed inputs (i.e., adversarial examples), which can mislead the agent to take suboptimal or unsafe actions. Recent methods improve attack effectiveness by leveraging future rewards t…

• Limits of Convergence-Rate Control for Open-Weight Safety

  Domenic Rosati, Xijie Zeng, Hong Huang, Sebastian Dionicio, Subhabrata Majumdar, Frank Rudzicz, Hassan Sajjad · 24. Februar 2026

  Open-weight foundation models can be fine-tuned for harmful purposes after release, yet no existing training resistance methods provide theoretical guarantees. Treating these interventions as convergence-rate control problems allows us to connect optimization speed to the spectral structure of model…

• The Power of Decaying Steps: Enhancing Attack Stability and Transferability for Sign-based Optimizers

  Wei Tao, Yang Dai, Jincai Huang, Qing Tao · 24. Februar 2026

  Crafting adversarial examples can be formulated as an optimization problem. While sign-based optimizers such as I-FGSM and MI-FGSM have become the de facto standard for the induced optimization problems, there still exist several unsolved problems in theoretical grounding and practical reliability e…

• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models

  Zhixiang Guo, Siyuan Liang, Andras Balogh, Noah Lunberry, Rong-Cheng Tu, Mark Jelasity, Dacheng Tao · 24. Februar 2026

  Generative world models (WMs) are increasingly used to synthesize controllable, sensor-conditioned driving videos, yet their reliance on physical priors exposes novel attack surfaces. In this paper, we present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model at…

• Leap+Verify: Regime-Adaptive Speculative Weight Prediction for Accelerating Neural Network Training

  Jeremy McEntire · 24. Februar 2026

  We introduce Leap+Verify, a framework that applies speculative execution -- predicting future model weights and validating predictions before acceptance -- to accelerate neural network training. Inspired by speculative decoding in language model inference and by the Automatically Scalable Computatio…

• Dirichlet Scale Mixture Priors for Bayesian Neural Networks

  August Arnstad, Leiv R{\o}nneberg, Geir Storvik · 24. Februar 2026

  Neural networks are the cornerstone of modern machine learning, yet can be difficult to interpret, give overconfident predictions and are vulnerable to adversarial attacks. Bayesian neural networks (BNNs) provide some alleviation of these limitations, but have problems of their own. The key step of …

• Quantifying Automation Risk in High-Automation AI Systems: A Bayesian Framework for Failure Propagation and Optimal Oversight

  Vishal Srivastava, Tanmay Sah · 24. Februar 2026

  Organizations across finance, healthcare, transportation, content moderation, and critical infrastructure are rapidly deploying highly automated AI systems, yet they lack principled methods to quantify how increasing automation amplifies harm when failures occur. We propose a parsimonious Bayesian r…

• Spilled Energy in Large Language Models

  Adrian Robert Minut, Hazem Dewidar, Iacopo Masi · 24. Februar 2026

  We reinterpret the final Large Language Model (LLM) softmax classifier as an Energy-Based Model (EBM), decomposing the sequence-to-sequence probability chain into multiple interacting EBMs at inference. This principled approach allows us to track "energy spills" during decoding, which we empirically…

• Hiding in Plain Text: Detecting Concealed Jailbreaks via Activation Disentanglement

  Amirhossein Farzam, Majid Behabahani, Mani Malek, Yuriy Nevmyvaka, Guillermo Sapiro · 24. Februar 2026

  Large language models (LLMs) remain vulnerable to jailbreak prompts that are fluent and semantically coherent, and therefore difficult to detect with standard heuristics. A particularly challenging failure mode occurs when an attacker tries to hide the malicious goal of their request by manipulating…

• BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models

  Xiaobei Yan, Yiming Li, Hao Wang, Han Qiu, Tianwei Zhang · 24. Februar 2026

  Large language models (LLMs) are widely deployed, but their substantial compute demands make them vulnerable to inference cost attacks that aim to deliberately maximize the output length. In this work, we investigate a distinct attack surface: maximizing inference cost by tampering with the model pa…

• Softmax is not Enough (for Adaptive Conformal Classification)

  Navid Akhavan Attar, Hesam Asadollahzadeh, Ling Luo, Uwe Aickelin · 24. Februar 2026

  The merit of Conformal Prediction (CP), as a distribution-free framework for uncertainty quantification, depends on generating prediction sets that are efficient, reflected in small average set sizes, while adaptive, meaning they signal uncertainty by varying in size according to input difficulty. A…