Adversarial Robustness in Machine Learning

Neueste Paper

• VERA: Variational Inference Framework for Jailbreaking Large Language Models

  Anamika Lochab, Lu Yan, Patrick Pynadath, Xiangyu Zhang, Ruqi Zhang · 7. November 2025

  The rise of API-only access to state-of-the-art LLMs highlights the need for effective black-box jailbreak methods to identify model vulnerabilities in real-world settings. Without a principled objective for gradient-based optimization, most existing approaches rely on genetic algorithms, which are …

• REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMs

  Liran Cohen, Yaniv Nemcovesky, Avi Mendelson · 7. November 2025

  Machine unlearning aims to remove the influence of specific training data from a model without requiring full retraining. This capability is crucial for ensuring privacy, safety, and regulatory compliance. Therefore, verifying whether a model has truly forgotten target data is essential for maintain…

• On Joint Regularization and Calibration in Deep Ensembles

  Laurits Fredsgaard (Department of Applied Mathematics and Computer Science, Technical University of Denmark), Mikkel N. Schmidt (Department of Applied Mathematics and Computer Science, Technical University of Denmark) · 7. November 2025

  Deep ensembles are a powerful tool in machine learning, improving both model performance and uncertainty calibration. While ensembles are typically formed by training and tuning models individually, evidence suggests that jointly tuning the ensemble can lead to better performance. This paper investi…

• From Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generation

  Najrin Sultana, Md Rafi Ur Rashid, Kang Gu, Shagufta Mehnaz · 6. November 2025

  LLMs can provide substantial zero-shot performance on diverse tasks using a simple task prompt, eliminating the need for training or fine-tuning. However, when applying these models to sensitive tasks, it is crucial to thoroughly assess their robustness against adversarial inputs. In this work, we i…

• Why Machine Learning Models Fail to Fully Capture Epistemic Uncertainty

  Sebasti\'an Jim\'enez, Mira J\"urgens, Willem Waegeman · 6. November 2025

  In recent years various supervised learning methods that disentangle aleatoric and epistemic uncertainty based on second-order distributions have been proposed. We argue that these methods fail to capture critical components of epistemic uncertainty, particularly due to the often-neglected component…

• NeuralSurv: Deep Survival Analysis with Bayesian Uncertainty Quantification

  M\'elodie Monod, Alessandro Micheli, Samir Bhatt · 6. November 2025

  We introduce NeuralSurv, the first deep survival model to incorporate Bayesian uncertainty quantification. Our non-parametric, architecture-agnostic framework captures time-varying covariate-risk relationships in continuous time via a novel two-stage data-augmentation scheme, for which we establish …

• Death by a Thousand Prompts: Open Model Vulnerability Analysis

  Amy Chang, Nicholas Conley, Harish Santhanalakshmi Ganesan, Adam Swanda · 6. November 2025

  Open-weight models provide researchers and developers with accessible foundations for diverse downstream applications. We tested the safety and security postures of eight open-weight large language models (LLMs) to identify vulnerabilities that may impact subsequent fine-tuning and deployment. Using…

• Adaptive and Robust Data Poisoning Detection and Sanitization in Wearable IoT Systems using Large Language Models

  W. K. M Mithsara, Ning Yang, Ahmed Imteaj, Hussein Zangoti, Abdur R. Shahid · 6. November 2025

  The widespread integration of wearable sensing devices in Internet of Things (IoT) ecosystems, particularly in healthcare, smart homes, and industrial applications, has required robust human activity recognition (HAR) techniques to improve functionality and user experience. Although machine learning…

• The Structure of Cross-Validation Error: Stability, Covariance, and Minimax Limits

  Ido Nachum, R\"udiger Urbanke, Thomas Weinberger · 6. November 2025

  Despite ongoing theoretical research on cross-validation (CV), many theoretical questions about CV remain widely open. This motivates our investigation into how properties of algorithm-distribution pairs can affect the choice for the number of folds in $k$-fold cross-validation. Our results consis…

• Feature compression is the root cause of adversarial fragility in neural network classifiers

  Jingchao Gao, Ziqing Lu, Raghu Mudumbai, Xiaodong Wu, Jirong Yi, Myung Cho, Catherine Xu, Hui Xie, Weiyu Xu · 5. November 2025

  In this paper, we uniquely study the adversarial robustness of deep neural networks (NN) for classification tasks against that of optimal classifiers. We look at the smallest magnitude of possible additive perturbations that can change a classifier's output. We provide a matrix-theoretic explanation…

• DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging

  Felix Wagner, Pramit Saha, Harry Anthony, J. Alison Noble, Konstantinos Kamnitsas · 5. November 2025

  Safe deployment of machine learning (ML) models in safety-critical domains such as medical imaging requires detecting inputs with characteristics not seen during training, known as out-of-distribution (OOD) detection, to prevent unreliable predictions. Effective OOD detection after deployment could …

• Verifying LLM Inference to Prevent Model Weight Exfiltration

  Roy Rinberg, Adam Karvonen, Alex Hoover, Daniel Reuter, Keri Warr · 5. November 2025

  As large AI models become increasingly valuable assets, the risk of model weight exfiltration from inference servers grows accordingly. An attacker controlling an inference server may exfiltrate model weights by hiding them within ordinary model outputs, a strategy known as steganography. This work …

• End-to-End Probabilistic Framework for Learning with Hard Constraints

  Utkarsh Utkarsh, Danielle C. Maddix, Ruijun Ma, Michael W. Mahoney, Yuyang Wang · 5. November 2025

  We present ProbHardE2E, a probabilistic forecasting framework that incorporates hard operational/physical constraints, and provides uncertainty quantification. Our methodology uses a novel differentiable probabilistic projection layer (DPPL) that can be combined with a wide range of neural network a…

• AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models

  Aashray Reddy, Andrew Zagula, Nicholas Saban · 5. November 2025

  Large Language Models (LLMs) remain vulnerable to jailbreaking attacks where adversarial prompts elicit harmful outputs, yet most evaluations focus on single-turn interactions while real-world attacks unfold through adaptive multi-turn conversations. We present AutoAdv, a training-free framework for…

• RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks

  Mian Wu, Gavin Zhang, Sewon Min, Sergey Levine, Aviral Kumar · 4. November 2025

  Open-ended generation tasks require outputs to satisfy diverse and often implicit task-specific evaluation rubrics. The sheer number of relevant rubrics leads to prohibitively high verification costs and incomplete assessments of a response, making reinforcement learning (RL) post-training with rubr…

• Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

  Kayua Oleques Paim, Rodrigo Brandao Mansilha, Diego Kreutz, Muriel Figueredo Franco, Weverton Cordeiro · 4. November 2025

  The rapid proliferation of Large Language Models (LLMs) has raised significant concerns about their security against adversarial attacks. In this work, we propose a novel approach to crafting universal jailbreaks and data extraction attacks by exploiting latent space discontinuities, an architectura…

• Partial Trace-Class Bayesian Neural Networks

  Arran Carter, Torben Sell · 4. November 2025

  Bayesian neural networks (BNNs) allow rigorous uncertainty quantification in deep learning, but often come at a prohibitive computational cost. We propose three different innovative architectures of partial trace-class Bayesian neural networks (PaTraC BNNs) that enable uncertainty quantification com…

• Probabilistic Robustness for Free? Revisiting Training via a Benchmark

  Yi Zhang, Zheng Wang, Chen Zhen, Wenjie Ruan, Qing Guo, Siddartha Khastgir, Carsten Maple, Xingyu Zhao · 4. November 2025

  Deep learning models are notoriously vulnerable to imperceptible perturbations. Most existing research centers on adversarial robustness (AR), which evaluates models under worst-case scenarios by examining the existence of deterministic adversarial examples (AEs). In contrast, probabilistic robustne…

• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness

  Longwei Wang, Ifrat Ikhtear Uddin, KC Santosh, Chaowei Zhang, Xiao Qin, Yang Zhou · 4. November 2025

  Adversarial examples reveal critical vulnerabilities in deep neural networks by exploiting their sensitivity to imperceptible input perturbations. While adversarial training remains the predominant defense strategy, it often incurs significant computational cost and may compromise clean-data accurac…

• Adversarial D\'ej\`a Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks

  Mahavir Dabas, Tran Huynh, Nikhil Reddy Billa, Jiachen T. Wang, Peng Gao, Charith Peris, Yao Ma, Rahul Gupta, Ming Jin, Prateek Mittal, Ruoxi Jia · 4. November 2025

  Large language models remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs. Defending against novel jailbreaks represents a critical challenge in AI safety. Adversarial training -- designed to make models robust against worst-case perturbations -- has been t…

• Lorica: A Synergistic Fine-Tuning Framework for Advancing Personalized Adversarial Robustness

  Tianyu Qi, Lei Xue, Yufeng Zhan, Xiaobo Ma · 4. November 2025

  The growing use of large pre-trained models in edge computing has made model inference on mobile clients both feasible and popular. Yet these devices remain vulnerable to adversarial attacks, threatening model robustness and security. Federated adversarial training (FAT) offers a promising solution …

• ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training

  Xin Yao, Haiyang Zhao, Yimin Chen, Jiawei Guo, Kecheng Huang, Ming Zhao · 4. November 2025

  The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from large-scale web data through self-supervised contrastive learning. Yet, its reliance on uncurated Internet-sourced data exposes it to data poisoning and backd…

• Keep It Real: Challenges in Attacking Compression-Based Adversarial Purification

  Samuel R\"aber, Till Aczel, Andreas Plesner, Roger Wattenhofer · 4. November 2025

  Previous work has suggested that preprocessing images through lossy compression can defend against adversarial perturbations, but comprehensive attack evaluations have been lacking. In this paper, we construct strong white-box and adaptive attacks against various compression models and identify a cr…

• Adversarial Distilled Retrieval-Augmented Guarding Model for Online Malicious Intent Detection

  Yihao Guo, Haocheng Bian, Liutong Zhou, Ze Wang, Zhaoyi Zhang, Francois Kawala, Milan Dean, Ian Fischer, Yuantao Peng, Noyan Tokgozoglu, Ivan Barrientos, Riyaaz Shaik, Rachel Li, Chandru Venkataraman, Reza Shifteh Far, Moses Pawar, Venkat Sundaranatha, Michael Xu, Frank Chu · 4. November 2025

  With the deployment of Large Language Models (LLMs) in interactive applications, online malicious intent detection has become increasingly critical. However, existing approaches fall short of handling diverse and complex user queries in real time. To address these challenges, we introduce ADRAG (Adv…

• Stable but Miscalibrated: A Kantian View on Overconfidence from Filters to Large Language Models

  Akira Okutomi · 4. November 2025

  We reinterpret Kant's Critique of Pure Reason as a theory of feedback stability, viewing reason as a regulator that keeps inference within the bounds of possible experience. We formalize this intuition via a composite instability index (H-Risk) combining spectral margin, conditioning, temporal sensi…