Adversarial Robustness in Machine Learning

Neueste Paper

• Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?

  Junjie Mu, Zonghao Ying, Zhekui Fan, Zonglei Jing, Yaoyuan Zhang, Zhengmin Yu, Wenxin Zhang, Quanchen Zou, Xiangzheng Zhang · 29. Januar 2026

  Jailbreak attacks on Large Language Models (LLMs) have demonstrated various successful methods whereby attackers manipulate models into generating harmful responses that they are designed to avoid. Among these, Greedy Coordinate Gradient (GCG) has emerged as a general and effective approach that opt…

• One Word is Enough: Minimal Adversarial Perturbations for Neural Text Ranking

  Tanmay Karmakar, Sourav Saha, Debapriyo Majumdar, Surjyanee Halder · 29. Januar 2026

  Neural ranking models (NRMs) achieve strong retrieval effectiveness, yet prior work has shown they are vulnerable to adversarial perturbations. We revisit this robustness question with a minimal, query-aware attack that promotes a target document by inserting or substituting a single, semantically a…

• Learning Contextual Runtime Monitors for Safe AI-Based Autonomy

  Alejandro Luque-Cerpa, Mengyuan Wang, Emil Carlsson, Sanjit A. Seshia, Devdatt Dubhashi, Hazem Torfah · 29. Januar 2026

  We introduce a novel framework for learning context-aware runtime monitors for AI-based control ensembles. Machine-learning (ML) controllers are increasingly deployed in (autonomous) cyber-physical systems because of their ability to solve complex decision-making tasks. However, their accuracy can d…

• Implicit Hypothesis Testing and Divergence Preservation in Neural Network Representations

  Kadircan Aksoy, Peter Jung, Protim Bhattacharjee · 29. Januar 2026

  We study the supervised training dynamics of neural classifiers through the lens of binary hypothesis testing. We model classification as a set of binary tests between class-conditional distributions of representations and empirically show that, along training trajectories, well-generalizing network…

• Self Voice Conversion as an Attack against Neural Audio Watermarking

  Yigitcan \"Ozer, Wanying Ge, Zhe Zhang, Xin Wang, Junichi Yamagishi · 29. Januar 2026

  Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and digital signal processing-based watermarking methods have improved imperceptibility and embedding capacity, robustness is…

• SemBind: Binding Diffusion Watermarks to Semantics Against Black-Box Forgery Attacks

  Xin Zhang, Zijin Yang, Kejiang Chen, Linfeng Ma, Weiming Zhang, Nenghai Yu · 29. Januar 2026

  Latent-based watermarks, integrated into the generation process of latent diffusion models (LDMs), simplify detection and attribution of generated images. However, recent black-box forgery attacks, where an attacker needs at least one watermarked image and black-box access to the provider's model, c…

• Feature-Space Adversarial Robustness Certification for Multimodal Large Language Models

  Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang · 29. Januar 2026

  Multimodal large language models (MLLMs) exhibit strong capabilities across diverse applications, yet remain vulnerable to adversarial perturbations that distort their feature representations and induce erroneous predictions. To address this vulnerability, we propose Feature-space Smoothing (FS), a …

• Doxing via the Lens: Revealing Location-related Privacy Leakage on Multi-modal Large Reasoning Models

  Weidi Luo, Tianyu Lu, Qiming Zhang, Xiaogeng Liu, Bin Hu, Yue Zhao, Jieyu Zhao, Song Gao, Patrick McDaniel, Zhen Xiang, Chaowei Xiao · 29. Januar 2026

  Recent advances in multi-modal large reasoning models (MLRMs) have shown significant ability to interpret complex visual content. While these models enable impressive reasoning capabilities, they also introduce novel and underexplored privacy risks. In this paper, we identify a novel category of pri…

• Membership Inference Attacks Against Fine-tuned Diffusion Language Models

  Yuetian Chen, Kaiyuan Zhang, Yuntao Du, Edoardo Stoppa, Charles Fleming, Ashish Kundu, Bruno Ribeiro, Ninghui Li · 29. Januar 2026

  Diffusion Language Models (DLMs) represent a promising alternative to autoregressive language models, using bidirectional masked token prediction. Yet their susceptibility to privacy leakage via Membership Inference Attacks (MIA) remains critically underexplored. This paper presents the first system…

• Reinforcement Unlearning via Group Relative Policy Optimization

  Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci · 29. Januar 2026

  During pretraining, LLMs inadvertently memorize sensitive or copyrighted data, posing significant compliance challenges under legal frameworks like the GDPR and the EU AI Act. Fulfilling these mandates demands techniques that can remove information from a deployed model without retraining from scrat…

• In-Context Bias Propagation in LLM-Based Tabular Data Generation

  Pol G. Recasens, Alberto Gutierrez, Jordi Torres, Josep. Ll Berral, Javier Carnerero-Cano, Anisa Halimi, Kieran Fraser · 29. Januar 2026

  Large Language Models (LLMs) are increasingly used for synthetic tabular data generation through in-context learning (ICL), offering a practical solution for data augmentation in data scarce scenarios. While prior work has shown the potential of LLMs to improve downstream task performance through au…

• Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning

  Xianglin Yang, Gelei Deng, Jieming Shi, Tianwei Zhang, Jin Song Dong · 29. Januar 2026

  Large language models (LLMs) are vital for a wide range of applications yet remain susceptible to jailbreak threats, which could lead to the generation of inappropriate responses. Conventional defenses, such as refusal and adversarial training, often fail to cover corner cases or rare domains, leavi…

• Benchmarking LLAMA Model Security Against OWASP Top 10 For LLM Applications

  Nourin Shahin, Izzat Alsmadi · 29. Januar 2026

  As large language models (LLMs) move from research prototypes to enterprise systems, their security vulnerabilities pose serious risks to data privacy and system integrity. This study benchmarks various Llama model variants against the OWASP Top 10 for LLM Applications framework, evaluating threat d…

• SABRE-FL: Selective and Accurate Backdoor Rejection for Federated Prompt Learning

  Momin Ahmad Khan, Yasra Chandio, Fatima Muhammad Anwar · 28. Januar 2026

  Federated Prompt Learning has emerged as a communication-efficient and privacy-preserving paradigm for adapting large vision-language models like CLIP across decentralized clients. However, the security implications of this setup remain underexplored. In this work, we present the first study of back…

• Selective Steering: Norm-Preserving Control Through Discriminative Layer Selection

  Quy-Anh Dang, Chris Ngo · 28. Januar 2026

  Despite significant progress in alignment, large language models (LLMs) remain vulnerable to adversarial attacks that elicit harmful behaviors. Activation steering techniques offer a promising inference-time intervention approach, but existing methods suffer from critical limitations: activation add…

• MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution

  Zihan Wu, Jie Xu, Yun Peng, Chun Yong Chong, Xiaohua Jia · 28. Januar 2026

  Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to two key limitations: the heterogeneity of vulnerability patterns undermines the effectiveness of a single unified model, and manual prompt engineering for massive weakness categories is unscalable. To addre…

• RvB: Automating AI System Hardening via Iterative Red-Blue Games

  Lige Huang, Zicheng Liu, Jie Zhang, Lewen Yan, Dongrui Liu, Jing Shao · 28. Januar 2026

  The dual offensive and defensive utility of Large Language Models (LLMs) highlights a critical gap in AI security: the lack of unified frameworks for dynamic, iterative adversarial adaptation hardening. To bridge this gap, we propose the Red Team vs. Blue Team (RvB) framework, formulated as a traini…

• Beyond Memorization: Selective Learning for Copyright-Safe Diffusion Model Training

  Divya Kothandaraman, Jaclyn Pytlarz · 28. Januar 2026

  Memorization in large-scale text-to-image diffusion models poses significant security and intellectual property risks, enabling adversarial attribute extraction and the unauthorized reproduction of sensitive or proprietary features. While conventional dememorization techniques, such as regularizatio…

• Concept activation vectors: a unifying view and adversarial attacks

  Ekkehard Schnoor, Malik Tiomoko, Jawher Said, Alex Jung, Wojciech Samek · 28. Januar 2026

  Concept Activation Vectors (CAVs) are a tool from explainable AI, offering a promising approach for understanding how human-understandable concepts are encoded in a model's latent spaces. They are computed from hidden-layer activations of inputs belonging either to a concept class or to non-concept …

• SHIELD: An Auto-Healing Agentic Defense Framework for LLM Resource Exhaustion Attacks

  Nirhoshan Sivaroopan, Kanchana Thilakarathna, Albert Zomaya, Manu, Yi Guo, Jo Plested, Tim Lynar, Jack Yang, Wangli Yang · 28. Januar 2026

  Sponge attacks increasingly threaten LLM systems by inducing excessive computation and DoS. Existing defenses either rely on statistical filters that fail on semantically meaningful attacks or use static LLM-based detectors that struggle to adapt as attack strategies evolve. We introduce SHIELD, a m…

• SETA: Statistical Fault Attribution for Compound AI Systems

  Sayak Chowdhury, Meenakshi D'Souza · 28. Januar 2026

  Modern AI systems increasingly comprise multiple interconnected neural networks to tackle complex inference tasks. Testing such systems for robustness and safety entails significant challenges. Current state-of-the-art robustness testing techniques, whether black-box or white-box, have been proposed…

• GAVEL: Towards rule-based safety through activation monitoring

  Shir Rozenfeld, Rahul Pankajakshan, Itay Zloczower, Eyal Lenga, Gilad Gressel, Yisroel Mirsky · 28. Januar 2026

  Large language models (LLMs) are increasingly paired with activation-based monitoring to detect and prevent harmful behaviors that may not be apparent at the surface-text level. However, existing activation safety approaches, trained on broad misuse datasets, struggle with poor precision, limited fl…

• From Internal Diagnosis to External Auditing: A VLM-Driven Paradigm for Online Test-Time Backdoor Defense

  Binyan Xu, Fan Yang, Xilin Dai, Di Tang, Kehuan Zhang · 28. Januar 2026

  Deep Neural Networks remain inherently vulnerable to backdoor attacks. Traditional test-time defenses largely operate under the paradigm of internal diagnosis methods like model repairing or input robustness, yet these approaches are often fragile under advanced attacks as they remain entangled with…

• Component-Aware Pruning Framework for Neural Network Controllers via Gradient-Based Importance Estimation

  Ganesh Sundaram, Jonas Ulmen, Daniel G\"orges · 28. Januar 2026

  The transition from monolithic to multi-component neural architectures in advanced neural network controllers poses substantial challenges due to the high computational complexity of the latter. Conventional model compression techniques for complexity reduction, such as structured pruning based on n…

• Robust Uncertainty Estimation under Distribution Shift via Difference Reconstruction

  Xinran Xu, Li Rong Wang, Xiuyi Fan · 28. Januar 2026

  Estimating uncertainty in deep learning models is critical for reliable decision-making in high-stakes applications such as medical imaging. Prior research has established that the difference between an input sample and its reconstructed version produced by an auxiliary model can serve as a useful p…