Adversarial Robustness in Machine Learning

Neueste Paper

• Model-Based Reward Shaping for Adversarial Inverse Reinforcement Learning in Stochastic Environments

  Simon Sinong Zhan, Philip Wang, Qingyuan Wu, Yixuan Wang, Ruochen Jiao, Chao Huang, Qi Zhu · 1. Dezember 2025

  In this paper, we aim to tackle the limitation of the Adversarial Inverse Reinforcement Learning (AIRL) method in stochastic environments where theoretical results cannot hold and performance is degraded. To address this issue, we propose a novel method which infuses the dynamics information into th…

• Curvature Dynamic Black-box Attack: revisiting adversarial robustness via dynamic curvature estimation

  Peiran Sun · 1. Dezember 2025

  Adversarial attack reveals the vulnerability of deep learning models. It is assumed that high curvature may give rise to rough decision boundary and thus result in less robust models. However, the most commonly used \textit{curvature} is the curvature of loss function, scores or other parameters fro…

• ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning

  Zhaorun Chen, Mintong Kang, Bo Li · 1. Dezember 2025

  Autonomous agents powered by foundation models have seen widespread adoption across various real-world applications. However, they remain highly vulnerable to malicious instructions and attacks, which can result in severe consequences such as privacy breaches and financial losses. More critically, e…

• The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning

  Ethan Hsu, Harry Chen, Chudi Zhong, Lesia Semenova · 1. Dezember 2025

  Real-world machine learning (ML) pipelines rarely produce a single model; instead, they produce a Rashomon set of many near-optimal ones. We show that this multiplicity reshapes key aspects of trustworthiness. At the individual-model level, sparse interpretable models tend to preserve privacy but ar…

• Breaking the Illusion: Consensus-Based Generative Mitigation of Adversarial Illusions in Multi-Modal Embeddings

  Fatemeh Akbarian, Anahita Baninajjar, Yingyi Zhang, Ananth Balashankar, Amir Aminifar · 1. Dezember 2025

  Multi-modal foundation models align images, text, and other modalities in a shared embedding space but remain vulnerable to adversarial illusions (Zhang et al., 2025), where imperceptible perturbations disrupt cross-modal alignment and mislead downstream tasks. To counteract the effects of adversari…

• NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks

  Lama Sleem, Jerome Francois, Lujun Li, Nathan Foucher, Niccolo Gentile, Radu State · 1. Dezember 2025

  Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To addre…

• iSeal: Encrypted Fingerprinting for Reliable LLM Ownership Verification

  Zixun Xiong, Gaoyi Wu, Qingyang Yu, Mingyu Derek Ma, Lingfeng Yao, Miao Pan, Xiaojiang Du, Hao Wang · 1. Dezember 2025

  Given the high cost of large language model (LLM) training from scratch, safeguarding LLM intellectual property (IP) has become increasingly crucial. As the standard paradigm for IP ownership verification, LLM fingerprinting thus plays a vital role in addressing this challenge. Existing LLM fingerpr…

• Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges

  Guanxi Lu, Hao Mark Chen, Zhiqiang Que, Wayne Luk, Hongxiang Fan · 1. Dezember 2025

  Large language models (LLMs) have shown promising performance across various tasks. However, their autoregressive decoding process poses significant challenges for efficient deployment on existing AI hardware. Quantization alleviates memory and compute pressure by compressing weights, activations, a…

• Network Inversion for Uncertainty-Aware Out-of-Distribution Detection

  Pirzada Suhail, Rehna Afroz, Gouranga Bala, Amit Sethi · 1. Dezember 2025

  Out-of-distribution (OOD) detection and uncertainty estimation (UE) are critical components for building safe machine learning systems, especially in real-world scenarios where unexpected inputs are inevitable. However the two problems have, until recently, separately been addressed. In this work, w…

• AgentShield: Make MAS more secure and efficient

  Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov, Jiong Lou, Jie LI · 1. Dezember 2025

  Large Language Model (LLM)-based Multi-Agent Systems (MAS) offer powerful cooperative reasoning but remain vulnerable to adversarial attacks, where compromised agents can undermine the system's overall performance. Existing defenses either depend on single trusted auditors, creating single points of…

• Accelerated Execution of Bayesian Neural Networks using a Single Probabilistic Forward Pass and Code Generation

  Bernhard Klein, Falk Selker, Hendrik Borras, Sophie Steger, Franz Pernkopf, Holger Fr\"oning · 1. Dezember 2025

  Machine learning models perform well across domains such as diagnostics, weather forecasting, NLP, and autonomous driving, but their limited uncertainty handling restricts use in safety-critical settings. Traditional neural networks often fail to detect out-of-domain (OOD) data and may output confid…

• RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attacks

  Yanping Li, Zhening Liu, Zijian Li, Zehong Lin, Jun Zhang · 1. Dezember 2025

  As a mainstream technique for 3D reconstruction, 3D Gaussian splatting (3DGS) has been applied in a wide range of applications and services. Recent studies have revealed critical vulnerabilities in this pipeline and introduced computation cost attacks that lead to malicious resource occupancies and …

• Decomposed Trust: Exploring Privacy, Adversarial Robustness, Fairness, and Ethics of Low-Rank LLMs

  Daniel Agyei Asante, Md Mokarram Chowdhury, Yang Li · 1. Dezember 2025

  Large language models (LLMs) have driven major advances across domains, yet their massive size hinders deployment in resource-constrained settings. Model compression addresses this challenge, with low-rank factorization emerging as a particularly effective method for reducing size, memory, and compu…

• Distillability of LLM Security Logic: Predicting Attack Success Rate of Outline Filling Attack via Ranking Regression

  Tianyu Zhang, Zihang Xi, Jingyu Hua, Sheng Zhong · 1. Dezember 2025

  In the realm of black-box jailbreak attacks on large language models (LLMs), the feasibility of constructing a narrow safety proxy, a lightweight model designed to predict the attack success rate (ASR) of adversarial prompts, remains underexplored. This work investigates the distillability of an LLM…

• ABLE: Using Adversarial Pairs to Construct Local Models for Explaining Model Predictions

  Krishna Khadka, Sunny Shree, Pujan Budhathoki, Yu Lei, Raghu Kacker, D. Richard Kuhn · 1. Dezember 2025

  Machine learning models are increasingly used in critical applications but are mostly "black boxes" due to their lack of transparency. Local explanation approaches, such as LIME, address this issue by approximating the behavior of complex models near a test instance using simple, interpretable model…

• Exploring Dynamic Properties of Backdoor Training Through Information Bottleneck

  Xinyu Liu, Xu Zhang, Can Chen, Ren Wang · 1. Dezember 2025

  Understanding how backdoor data influences neural network training dynamics remains a complex and underexplored challenge. In this paper, we present a rigorous analysis of the impact of backdoor data on the learning process, with a particular focus on the distinct behaviors between the target class …

• Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification

  Yanxi Li, Ruocheng Shan · 1. Dezember 2025

  Large language models are increasingly used for text classification tasks such as sentiment analysis, yet their reliance on natural language prompts exposes them to prompt injection attacks. In particular, class-directive injections exploit knowledge of the model's label set (e.g., positive vs. nega…

• A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Reality

  Rongqian Chen, Allison Andreyev, Yanming Xiu, Joshua Chilukuri, Shunav Sen, Mahdi Imani, Bin Li, Maria Gorlatova, Gang Tan, Tian Lan · 1. Dezember 2025

  Augmented Reality (AR) enriches human perception by overlaying virtual elements onto the physical world. However, this tight coupling between virtual and real content makes AR vulnerable to cognitive attacks: manipulations that distort users' semantic understanding of the environment. Existing detec…

• Dataset Poisoning Attacks on Behavioral Cloning Policies

  Akansha Kalra, Soumil Datta, Ethan Gilmore, Duc La, Guanhong Tao, Daniel S. Brown · 27. November 2025

  Behavior Cloning (BC) is a popular framework for training sequential decision policies from expert demonstrations via supervised learning. As these policies are increasingly being deployed in the real world, their robustness and potential vulnerabilities are an important concern. In this work, we pe…

• Geometric Calibration and Neutral Zones for Uncertainty-Aware Multi-Class Classification

  Soumojit Das, Nairanjana Dasgupta, Prashanta Dutta · 27. November 2025

  Modern artificial intelligence systems make critical decisions yet often fail silently when uncertain. We develop a geometric framework for post-hoc calibration of neural network probability outputs, treating probability vectors as points on the $(c-1)$-dimensional probability simplex equipped with …

• TAB-DRW: A DFT-based Robust Watermark for Generative Tabular Data

  Yizhou Zhao, Xiang Li, Peter Song, Qi Long, Weijie Su · 27. November 2025

  The rise of generative AI has enabled the production of high-fidelity synthetic tabular data across fields such as healthcare, finance, and public policy, raising growing concerns about data provenance and misuse. Watermarking offers a promising solution to address these concerns by ensuring the tra…

• Beyond Accuracy: An Empirical Study of Uncertainty Estimation in Imputation

  Zarin Tahia Hossain, Mostafa Milani · 27. November 2025

  Handling missing data is a central challenge in data-driven analysis. Modern imputation methods not only aim for accurate reconstruction but also differ in how they represent and quantify uncertainty. Yet, the reliability and calibration of these uncertainty estimates remain poorly understood. This …

• HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks

  Raffaele Mura, Giuseppe Floris, Luca Scionis, Giorgio Piras, Maura Pintor, Ambra Demontis, Giorgio Giacinto, Battista Biggio, Fabio Roli · 27. November 2025

  Gradient-based attacks are a primary tool to evaluate robustness of machine-learning models. However, many attacks tend to provide overly-optimistic evaluations as they use fixed loss functions, optimizers, step-size schedulers, and default hyperparameters. In this work, we tackle these limitations …

• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training

  Erh-Chung Chen, Che-Rung Lee · 27. November 2025

  Adversarial training is a widely adopted strategy to bolster the robustness of neural network models against adversarial attacks. This paper revisits the fundamental assumptions underlying image classification and suggests that representing data as one-hot labels is a key factor that leads to vulner…

• Dynamic Epsilon Scheduling: A Multi-Factor Adaptive Perturbation Budget for Adversarial Training

  Alan Mitkiy, James Smith, Myungseo wong, Hana Satou, Hiroshi Tanaka, Emily Johnson · 27. November 2025

  Adversarial training is among the most effective strategies for defending deep neural networks against adversarial examples. A key limitation of existing adversarial training approaches lies in their reliance on a fixed perturbation budget, which fails to account for instance-specific robustness cha…