Adversarial Robustness in Machine Learning

Neueste Paper

• Disrupting Hierarchical Reasoning: Adversarial Protection for Geographic Privacy in Multimodal Reasoning Models

  Jiaming Zhang, Che Wang, Yang Cao, Longtao Huang, Wei Yang Bryan Lim · 10. Dezember 2025

  Multi-modal large reasoning models (MLRMs) pose significant privacy risks by inferring precise geographic locations from personal images through hierarchical chain-of-thought reasoning. Existing privacy protection techniques, primarily designed for perception-based models, prove ineffective against …

• Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning

  Junnan Qiu, Jie Li · 10. Dezember 2025

  Offline Reinforcement Learning (RL) enables policy optimization from static datasets but is inherently vulnerable to data poisoning attacks. Existing attack strategies typically rely on locally uniform perturbations, which treat all samples indiscriminately. This approach is inefficient, as it waste…

• Interpreting Structured Perturbations in Image Protection Methods for Diffusion Models

  Michael R. Martin, Garrick Chan, Kwan-Liu Ma · 10. Dezember 2025

  Recent image protection mechanisms such as Glaze and Nightshade introduce imperceptible, adversarially designed perturbations intended to disrupt downstream text-to-image generative models. While their empirical effectiveness is known, the internal structure, detectability, and representational beha…

• OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities

  Sahil Verma, Keegan Hines, Jeff Bilmes, Charlotte Siska, Luke Zettlemoyer, Hila Gonen, Chandan Singh · 10. Dezember 2025

  The emerging capabilities of large language models (LLMs) have sparked concerns about their immediate potential for harmful misuse. The core approach to mitigate these concerns is the detection of harmful queries to the model. Current detection approaches are fallible, and are particularly susceptib…

• PrivTune: Efficient and Privacy-Preserving Fine-Tuning of Large Language Models via Device-Cloud Collaboration

  Yi Liu, Weixiang Han, Chengjun Cai, Xingliang Yuan, Cong Wang · 10. Dezember 2025

  With the rise of large language models, service providers offer language models as a service, enabling users to fine-tune customized models via uploaded private datasets. However, this raises concerns about sensitive data leakage. Prior methods, relying on differential privacy within device-cloud co…

• DASH: A Meta-Attack Framework for Synthesizing Effective and Stealthy Adversarial Examples

  Abdullah Al Nomaan Nafi, Habibur Rahaman, Zafaryab Haider, Tanzim Mahfuz, Fnu Suya, Swarup Bhunia, Prabuddha Chakraborty · 10. Dezember 2025

  Numerous techniques have been proposed for generating adversarial examples in white-box settings under strict Lp-norm constraints. However, such norm-bounded examples often fail to align well with human perception, and only recently have a few methods begun specifically exploring perceptually aligne…

• Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models

  Huzaifa Arif, Pin-Yu Chen, Alex Gittens, James Diffenderfer, Bhavya Kailkhura · 10. Dezember 2025

  With the increasing reliance on AI models for weather forecasting, it is imperative to evaluate their vulnerability to adversarial perturbations. This work introduces Weather Adaptive Adversarial Perturbation Optimization (WAAPO), a novel framework for generating targeted adversarial perturbations t…

• Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem

  Shiva Gaire, Srijan Gyawali, Saroj Mishra, Suman Niroula, Dilip Thakur, Umesh Yadav · 10. Dezember 2025

  The Model Context Protocol (MCP) has emerged as the de facto standard for connecting Large Language Models (LLMs) to external data and tools, effectively functioning as the "USB-C for Agentic AI." While this decoupling of context and execution solves critical interoperability challenges, it introduc…

• Improving the Sensitivity of Backdoor Detectors via Class Subspace Orthogonalization

  Guangmingmei Yang, David J. Miller, George Kesidis · 10. Dezember 2025

  Most post-training backdoor detection methods rely on attacked models exhibiting extreme outlier detection statistics for the target class of an attack, compared to non-target classes. However, these approaches may fail: (1) when some (non-target) classes are easily discriminable from all others, in…

• Developing Distance-Aware Uncertainty Quantification Methods in Physics-Guided Neural Networks for Reliable Bearing Health Prediction

  Waleed Razzaq, Yun-Bo Zhao · 10. Dezember 2025

  Accurate and uncertainty-aware degradation estimation is essential for predictive maintenance in safety-critical systems like rotating machinery with rolling-element bearings. Many existing uncertainty methods lack confidence calibration, are costly to run, are not distance-aware, and fail to genera…

• Robust Agents in Open-Ended Worlds

  Mikayel Samvelyan · 10. Dezember 2025

  The growing prevalence of artificial intelligence (AI) in various applications underscores the need for agents that can successfully navigate and adapt to an ever-changing, open-ended world. A key challenge is ensuring these AI agents are robust, excelling not only in familiar settings observed duri…

• A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties

  Jinghao Wang, Ping Zhang, Carter Yagemann · 10. Dezember 2025

  Medical Large Language Models (LLMs) are increasingly deployed for clinical decision support across diverse specialties, yet systematic evaluation of their robustness to adversarial misuse and privacy leakage remains inaccessible to most researchers. Existing security benchmarks require GPU clusters…

• When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation

  Joshua Ward, Bochao Gu, Chi-Hua Wang, Guang Cheng · 10. Dezember 2025

  Large Language Models (LLMs) have recently demonstrated remarkable performance in generating high-quality tabular synthetic data. In practice, two primary approaches have emerged for adapting LLMs to tabular data generation: (i) fine-tuning smaller models directly on tabular datasets, and (ii) promp…

• Toward Reliable Machine Unlearning: Theory, Algorithms, and Evaluation

  Ali Ebrahimpour-Boroojeny · 9. Dezember 2025

  We propose new methodologies for both unlearning random set of samples and class unlearning and show that they outperform existing methods. The main driver of our unlearning methods is the similarity of predictions to a retrained model on both the forget and remain samples. We introduce Adversarial …

• Pay Less Attention to Function Words for Free Robustness of Vision-Language Models

  Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Chao Shen · 9. Dezember 2025

  To address the trade-off between robustness and performance for robust VLM, we observe that function words could incur vulnerability of VLMs against cross-modal adversarial attacks, and propose Function-word De-Attention (FDA) accordingly to mitigate the impact of function words. Similar to differen…

• Unlearning Inversion Attacks for Graph Neural Networks

  Jiahao Zhang, Yilong Wang, Zhiwei Zhang, Xiaorui Liu, Suhang Wang · 9. Dezember 2025

  Graph unlearning methods aim to efficiently remove the impact of sensitive data from trained GNNs without full retraining, assuming that deleted information cannot be recovered. In this work, we challenge this assumption by introducing the graph unlearning inversion attack: given only black-box acce…

• Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets

  Nabil Alami, Jad Zakharia, Souhaib Ben Taieb · 9. Dezember 2025

  Access to multiple predictive models trained for the same task, whether in regression or classification, is increasingly common in many applications. Aggregating their predictive uncertainties to produce reliable and efficient uncertainty quantification is therefore a critical but still underexplore…

• ThinkTrap: Denial-of-Service Attacks against Black-box LLM Services via Infinite Thinking

  Yunzhe Li, Jianan Wang, Hongzi Zhu, James Lin, Shan Chang, Minyi Guo · 9. Dezember 2025

  Large Language Models (LLMs) have become foundational components in a wide range of applications, including natural language understanding and generation, embodied intelligence, and scientific discovery. As their computational requirements continue to grow, these models are increasingly deployed as …

• AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing

  Ziming Hong, Tianyu Huang, Runnan Chen, Shanshan Ye, Mingming Gong, Bo Han, Tongliang Liu · 9. Dezember 2025

  Recent studies have extended diffusion-based instruction-driven 2D image editing pipelines to 3D Gaussian Splatting (3DGS), enabling faithful manipulation of 3DGS assets and greatly advancing 3DGS content creation. However, it also exposes these assets to serious risks of unauthorized editing and ma…

• Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Models

  Kassoum Sanogo, Renzo Ardiccioni · 9. Dezember 2025

  Vision-language models (VLMs) frequently generate hallucinated content plausible but incorrect claims about image content. We propose a training-free self-correction framework enabling VLMs to iteratively refine responses through uncertainty-guided visual re-attention. Our method combines multidimen…

• RL-MTJail: Reinforcement Learning for Automated Black-Box Multi-Turn Jailbreaking of Large Language Models

  Xiqiao Xiong, Ouxiang Li, Zhuo Liu, Moxin Li, Wentao Shi, Fuli Feng, Xiangnan He · 9. Dezember 2025

  Large language models are vulnerable to jailbreak attacks, threatening their safe deployment in real-world applications. This paper studies black-box multi-turn jailbreaks, aiming to train attacker LLMs to elicit harmful content from black-box models through a sequence of prompt-output interactions.…

• UncertaintyZoo: A Unified Toolkit for Quantifying Predictive Uncertainty in Deep Learning Systems

  Xianzong Wu, Xiaohong Li, Lili Quan, Qiang Hu · 9. Dezember 2025

  Large language models(LLMs) are increasingly expanding their real-world applications across domains, e.g., question answering, autonomous driving, and automatic software development. Despite this achievement, LLMs, as data-driven systems, often make incorrect predictions, which can lead to potential…

• Distribution-informed Online Conformal Prediction

  Dongjian Hu, Junxi Wu, Shu-Tao Xia, Changliang Zou · 9. Dezember 2025

  Conformal prediction provides a pivotal and flexible technique for uncertainty quantification by constructing prediction sets with a predefined coverage rate. Many online conformal prediction methods have been developed to address data distribution shifts in fully adversarial environments, resulting…

• Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning

  Lijun Zhang, Xiao Liu, Kaleel Mahmood, Caiwen Ding, Hui Guan · 9. Dezember 2025

  Visual content understanding frequently relies on multi-task models to extract robust representations of a single visual input for multiple downstream tasks. However, in comparison to extensively studied single-task models, the adversarial robustness of multi-task models has received significantly l…

• Exploring Adversarial Watermarking in Transformer-Based Models: Transferability and Robustness Against Defense Mechanism for Medical Images

  Rifat Sadik, Tanvir Rahman, Arpan Bhattacharjee, Bikash Chandra Halder, Ismail Hossain, Rifat Sarker Aoyon, Md. Golam Rabiul Alam, Jia Uddin · 9. Dezember 2025

  Deep learning models have shown remarkable success in dermatological image analysis, offering potential for automated skin disease diagnosis. Previously, convolutional neural network(CNN) based architectures have achieved immense popularity and success in computer vision (CV) based task like skin im…