Adversarial Robustness in Machine Learning

Neueste Paper

• Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?

  Kamalasankari Subramaniakuppusamy, Jugal Gajjar · 6. April 2026

  Post-hoc feature attribution methods are widely deployed in safety-critical vision systems, yet their stability under realistic input perturbations remains poorly characterized. Existing metrics evaluate explanations primarily under additive noise, collapse stability to a single scalar, and fail to …

• AgenticRed: Evolving Agentic Systems for Red-Teaming

  Jiayi Yuan, Jonathan N\"other, Natasha Jaques, Goran Radanovi\'c · 6. April 2026

  While recent automated red-teaming methods show promise for systematically exposing model vulnerabilities, most existing approaches rely on human-specified workflows. This dependence on manually designed workflows suffers from human biases and makes exploring the broader design space expensive. We i…

• Generalization Limits of Reinforcement Learning Alignment

  Haruhi Shida, Koo Imai, Keigo Kansa · 6. April 2026

  The safety of large language models (LLMs) relies on alignment techniques such as reinforcement learning from human feedback (RLHF). However, recent theoretical analyses suggest that reinforcement learning-based training does not acquire new capabilities but merely redistributes the utilization prob…

• Characterizing WebGPU Dispatch Overhead for LLM Inference Across Four GPU Vendors, Three Backends, and Three Browsers

  J\k{e}drzej Maczan · 6. April 2026

  WebGPU's security-focused design imposes per-operation validation that compounds across the many small dispatches in neural network inference, yet the true cost of this overhead is poorly characterized. We present a systematic characterization of WebGPU dispatch overhead for LLM inference at batch s…

• Chain-of-Authorization: Embedding authorization into large language models

  Yang Li, Yule Liu, Xinlei He, Youjian Zhao, Qi Li, Ke Xu · 6. April 2026

  Although Large Language Models (LLMs) have evolved from text generators into the cognitive core of modern AI systems, their inherent lack of authorization awareness exposes these systems to catastrophic risks, ranging from unintentional data leakage to unauthorized command execution. Existing defens…

• AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents

  Yunhao Feng, Yifan Ding, Yingshui Tan, Xingjun Ma, Yige Li, Yutao Wu, Yifeng Gao, Kun Zhai, Yanming Guo · 6. April 2026

  Computer-use agents extend language models from text generation to persistent action over tools, files, and execution environments. Unlike chat systems, they maintain state across interactions and translate intermediate outputs into concrete actions. This creates a distinct safety challenge in that …

• Modeling and Controlling Deployment Reliability under Temporal Distribution Shift

  Naimur Rahman, Naazreen Tabassum · 6. April 2026

  Machine learning models deployed in non-stationary environments are exposed to temporal distribution shift, which can erode predictive reliability over time. While common mitigation strategies such as periodic retraining and recalibration aim to preserve performance, they typically focus on average …

• Understanding the Effects of Safety Unalignment on Large Language Models

  John T. Halloran · 6. April 2026

  Safety alignment has become a critical step to ensure LLMs refuse harmful requests while providing helpful and harmless responses. However, despite the ubiquity of safety alignment for deployed frontier models, two separate lines of recent work--jailbreak-tuning (JT) and weight orthogonalization (WO…

• On the Extreme Variance of Certified Local Robustness Across Model Seeds

  Minh Le, Phuong Cao · 6. April 2026

  Robustness verification of neural networks, referring to formally proving that neural networks satisfy robustness properties, is of crucial importance in safety-critical applications, where model failures can result in loss of human life or million-dollar damages. However, the dependability of verif…

• Backdoor Attacks on Decentralised Post-Training

  O\u{g}uzhan Ersoy, Nikolay Blagoev, Jona te Lintelo, Stefanos Koffas, Marina Kr\v{c}ek, Stjepan Picek · 6. April 2026

  Decentralised post-training of large language models utilises data and pipeline parallelism techniques to split the data and the model. Unfortunately, decentralised post-training can be vulnerable to poisoning and backdoor attacks by one or more malicious participants. There have been several works …

• Beyond Semantic Manipulation: Token-Space Attacks on Reward Models

  Yuheng Zhang, Mingyue Huo, Minghao Zhu, Mengxue Zhang, Nan Jiang · 6. April 2026

  Reward models (RMs) are widely used as optimization targets in reinforcement learning from human feedback (RLHF), yet they remain vulnerable to reward hacking. Existing attacks mainly operate within the semantic space, constructing human-readable adversarial outputs that exploit RM biases. In this w…

• Safety, Security, and Cognitive Risks in World Models

  Manoj Parmar · 3. April 2026

  World models -- learned internal simulators of environment dynamics -- are rapidly becoming foundational to autonomous decision-making in robotics, autonomous vehicles, and agentic AI. Yet this predictive power introduces a distinctive set of safety, security, and cognitive risks. Adversaries can co…

• DiffGradCAM: A Universal Class Activation Map Resistant to Adversarial Training

  Jacob Piland, Chris Sweet, Adam Czajka · 3. April 2026

  Class Activation Mapping (CAM) and its gradient-based variants (e.g., GradCAM) have become standard tools for explaining Convolutional Neural Network (CNN) predictions. However, these approaches typically focus on individual logits, while for neural networks using softmax, the class membership proba…

• Towards Trustworthy Wi-Fi CSI-based Sensing: Systematic Evaluation of Adversarial Robustness

  Shreevanth Krishnaa Gopalakrishnan, Stephen Hailes · 3. April 2026

  Machine learning drives Channel State Information (CSI)-based human sensing in modern wireless networks, enabling applications like device-free human activity recognition (HAR) and identification (HID). However, the susceptibility of these models to adversarial perturbations raises security concerns…

• Safer by Diffusion, Broken by Context: Diffusion LLM's Safety Blessing and Its Failure Mode

  Zeyuan He, Yupeng Chen, Lang Lin, Yihan Wang, Shenxu Chang, Eric Sommerlade, Philip Torr, Junchi Yu, Adel Bibi, Jialin Yu · 3. April 2026

  Diffusion large language models (D-LLMs) offer an alternative to autoregressive LLMs (AR-LLMs) and have demonstrated advantages in generation efficiency. Beyond the utility benefits, we argue that D-LLMs exhibit a previously underexplored safety blessing: their diffusion-style generation confers int…

• Automated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks

  Anubhab Sahu, Diptisha Samanta, Reza Soosahabi · 2. April 2026

  System Instructions in Large Language Models (LLMs) are commonly used to enforce safety policies, define agent behavior, and protect sensitive operational context in agentic AI applications. These instructions may contain sensitive information such as API credentials, internal policies, and privileg…

• WARP: Guaranteed Inner-Layer Repair of NLP Transformers

  Hsin-Ling Hsu, Min-Yu Chen, Nai-Chia Chen, Yan-Ru Chen, Yi-Ling Chang, Fang Yu · 2. April 2026

  Transformer-based NLP models remain vulnerable to adversarial perturbations, yet existing repair methods face a fundamental trade-off: gradient-based approaches offer flexibility but lack verifiability and often overfit; methods that do provide repair guarantees are restricted to the final layer or …

• The Persistent Vulnerability of Aligned AI Systems

  Aengus Lynch · 2. April 2026

  Autonomous AI agents are being deployed with filesystem access, email control, and multi-step planning. This thesis contributes to four open problems in AI safety: understanding dangerous internal computations, removing dangerous behaviors once embedded, testing for vulnerabilities before deployment…

• Shapley-Guided Neural Repair Approach via Derivative-Free Optimization

  Xinyu Sun, Wanwei Liu, Haoang Chi, Tingyu Chen, Xiaoguang Mao, Shangwen Wang, Lei Bu, Jingyi Wang, Yang Tan, Zhenyi Qi · 2. April 2026

  DNNs are susceptible to defects like backdoors, adversarial attacks, and unfairness, undermining their reliability. Existing approaches mainly involve retraining, optimization, constraint-solving, or search algorithms. However, most methods rely on gradient calculations, restricting applicability to…

• Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning

  Swapnil Parekh · 2. April 2026

  A new generation of language models reasons entirely in continuous hidden states, producing no tokens and leaving no audit trail. We show that this silence creates a fundamentally new attack surface. ThoughtSteer perturbs a single embedding vector at the input layer; the model's own multi-pass r…

• Adversarial Moral Stress Testing of Large Language Models

  Saeid Jamshidi, Foutse Khomh, Arghavan Moradi Dakhel, Amin Nikanjam, Mohammad Hamdaqa, Kawser Wazed Nafi · 2. April 2026

  Evaluating the ethical robustness of large language models (LLMs) deployed in software systems remains challenging, particularly under sustained adversarial user interaction. Existing safety benchmarks typically rely on single-round evaluations and aggregate metrics, such as toxicity scores and refu…

• Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

  Zixiang Peng, Yongxiu Xu, Qinyi Zhang, Jiexun Shen, Yifan Zhang, Hongbo Xu, Yubin Wang, Gaopeng Gou · 2. April 2026

  Unified Multimodal Large Models (UMLMs) integrate understanding and generation capabilities within a single architecture. While this architectural unification, driven by the deep fusion of multimodal features, enhances model performance, it also introduces important yet underexplored safety challeng…

• Empirical Validation of the Classification-Verification Dichotomy for AI Safety Gates

  Arsenios Scrivens · 2. April 2026

  Can classifier-based safety gates maintain reliable oversight as AI systems improve over hundreds of iterations? We provide comprehensive empirical evidence that they cannot. On a self-improving neural controller (d=240), eighteen classifier configurations -- spanning MLPs, SVMs, random forests, k-N…

• QUEST: A robust attention formulation using query-modulated spherical attention

  Hariprasath Govindarajan, Per Sid\'en, Jacob Roll, Fredrik Lindsten · 2. April 2026

  The Transformer model architecture has become one of the most widely used in deep learning and the attention mechanism is at its core. The standard attention formulation uses a softmax operation applied to a scaled dot product between query and key vectors. We explore the role played by norms of the…

• Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

  Aaron Rose, Carissa Cullen, Brandon Gary Kaplowitz, Christian Schroeder de Witt · 2. April 2026

  As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise for detecting deception in single-agent settings, collusion is inherently a multi-…