Adversarial Robustness in Machine Learning

Neueste Paper

• Unsupervised Robust Domain Adaptation: Paradigm, Theory and Algorithm

  Fuxiang Huang, Xiaowei Fu, Shiyu Ye, Lina Ma, Wen Li, Xinbo Gao, David Zhang, Lei Zhang · 17. November 2025

  Unsupervised domain adaptation (UDA) aims to transfer knowledge from a label-rich source domain to an unlabeled target domain by addressing domain shifts. Most UDA approaches emphasize transfer ability, but often overlook robustness against adversarial attacks. Although vanilla adversarial training …

• HealSplit: Towards Self-Healing through Adversarial Distillation in Split Federated Learning

  Yuhan Xie, Chen Lyu · 17. November 2025

  Split Federated Learning (SFL) is an emerging paradigm for privacy-preserving distributed learning. However, it remains vulnerable to sophisticated data poisoning attacks targeting local features, labels, smashed data, and model weights. Existing defenses, primarily adapted from traditional Federate…

• The Map of Misbelief: Tracing Intrinsic and Extrinsic Hallucinations Through Attention Patterns

  Elyes Hajji, Aymen Bouguerra, Fabio Arnez · 17. November 2025

  Large Language Models (LLMs) are increasingly deployed in safety-critical domains, yet remain susceptible to hallucinations. While prior works have proposed confidence representation methods for hallucination detection, most of these approaches rely on computationally expensive sampling strategies a…

• Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition

  Barproda Halder, Faisal Hamman, Pasan Dissanayake, Qiuyi Zhang, Ilia Sucholutsky, Sanghamitra Dutta · 17. November 2025

  Spuriousness arises when there is an association between two or more variables in a dataset that are not causally related. In this work, we propose an explainability framework to preemptively disentangle the nature of such spurious associations in a dataset before model training. We leverage a body …

• On the Relationship Between Adversarial Robustness and Decision Region in Deep Neural Networks

  Seongjin Park, Haedong Jeong, Tair Djanibekov, Giyoung Jeon, Jinseok Seol, Jaesik Choi · 17. November 2025

  In general, Deep Neural Networks (DNNs) are evaluated by the generalization performance measured on unseen data excluded from the training phase. Along with the development of DNNs, the generalization performance converges to the state-of-the-art and it becomes difficult to evaluate DNNs solely base…

• PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks

  Zihan Liu, Jiayi Wen, Junru Wu, Xuyang Zou, Shouhong Tan, Zhirun Zheng, Cheng Huang · 17. November 2025

  In this paper, we introduce PrivDFS, a distributed feature-sharing framework for input-private inference in image classification. A single holistic intermediate representation in split inference gives diffusion-based Data Reconstruction Attacks (DRAs) sufficient signal to reconstruct the input with …

• Application-Specific Component-Aware Structured Pruning of Deep Neural Networks in Control via Soft Coefficient Optimization

  Ganesh Sundaram, Jonas Ulmen, Amjad Haider, Daniel G\"orges · 14. November 2025

  Deep neural networks (DNNs) offer significant flexibility and robust performance. This makes them ideal for building not only system models but also advanced neural network controllers (NNCs). However, their high complexity and computational needs often limit their use. Various model compression str…

• SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning

  Tairan Huang, Yulin Jin, Junxu Liu, Qingqing Ye, Haibo Hu · 14. November 2025

  Visual reinforcement learning has achieved remarkable progress in visual control and robotics, but its vulnerability to adversarial perturbations remains underexplored. Most existing black-box attacks focus on vector-based or discrete-action RL, and their effectiveness on image-based continuous cont…

• Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO

  Nikolay Blagoev, O\u{g}uzhan Ersoy, Lydia Yiyu Chen · 14. November 2025

  Group Relative Policy Optimization (GRPO) has demonstrated great utilization in post-training of Large Language Models (LLMs). In GRPO, prompts are answered by the model and, through reinforcement learning, preferred completions are learnt. Owing to the small communication volume, GRPO is inherently…

• Unlearning Imperative: Securing Trustworthy and Responsible LLMs through Engineered Forgetting

  James Jin Kang, Dang Bui, Thanh Pham, Huo-Chong Ling · 14. November 2025

  The growing use of large language models in sensitive domains has exposed a critical weakness: the inability to ensure that private information can be permanently forgotten. Yet these systems still lack reliable mechanisms to guarantee that sensitive information can be permanently removed once it ha…

• eXIAA: eXplainable Injections for Adversarial Attack

  Leonardo Pesce, Jiawen Wei, Gianmarco Mengaldo · 14. November 2025

  Post-hoc explainability methods are a subset of Machine Learning (ML) that aim to provide a reason for why a model behaves in a certain way. In this paper, we show a new black-box model-agnostic adversarial attack for post-hoc explainable Artificial Intelligence (XAI), particularly in the image doma…

• Tight Robustness Certification through the Convex Hull of $\ell_0$ Attacks

  Yuval Shapira, Dana Drachsler-Cohen · 14. November 2025

  Few-pixel attacks mislead a classifier by modifying a few pixels of an image. Their perturbation space is an $\ell_0$-ball, which is not convex, unlike $\ell_p$-balls for $p\geq1$. However, existing local robustness verifiers typically scale by relying on linear bound propagation, which captures con…

• Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback

  Qiwei Di, Jiafan He, Quanquan Gu · 13. November 2025

  Learning from human feedback plays an important role in aligning generative models, such as large language models (LLM). However, the effectiveness of this approach can be influenced by adversaries, who may intentionally provide misleading preferences to manipulate the output in an undesirable or ha…

• P-DRUM: Post-hoc Descriptor-based Residual Uncertainty Modeling for Machine Learning Potentials

  Shih-Peng Huang, Nontawat Charoenphakdee, Yuta Tsuboi, Yong-Bin Zhuang, Wenwen Li · 13. November 2025

  Ensemble method is considered the gold standard for uncertainty quantification (UQ) in machine learning interatomic potentials (MLIPs). However, their high computational cost can limit its practicality. Alternative techniques, such as Monte Carlo dropout and deep kernel learning, have been proposed …

• Abstract Gradient Training: A Unified Certification Framework for Data Poisoning, Unlearning, and Differential Privacy

  Philip Sosnin, Matthew Wicker, Josh Collyer, Calvin Tsay · 13. November 2025

  The impact of inference-time data perturbation (e.g., adversarial attacks) has been extensively studied in machine learning, leading to well-established certification techniques for adversarial robustness. In contrast, certifying models against training data perturbations remains a relatively under-…

• Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment

  Shigeki Kusaka, Keita Saito, Mikoto Kudo, Takumi Tanabe, Akifumi Wachi, Youhei Akimoto · 13. November 2025

  Large language models (LLMs) are increasingly deployed in real-world systems, making it critical to understand their vulnerabilities. While data poisoning attacks during RLHF/DPO alignment have been studied empirically, their theoretical foundations remain unclear. We investigate the minimum-cost po…

• Consensus Sampling for Safer Generative AI

  Adam Tauman Kalai, Yael Tauman Kalai, Or Zamir · 13. November 2025

  Many approaches to AI safety rely on inspecting model outputs or activations, yet certain risks are inherently undetectable by inspection alone. We propose a complementary, architecture-agnostic approach that enhances safety through the aggregation of multiple generative models, with the aggregated …

• Boosting Adversarial Transferability via Ensemble Non-Attention

  Yipeng Zou, Qin Liu, Jie Wu, Yu Peng, Guo Chen, Hui Zhou, Guanghui Ye · 13. November 2025

  Ensemble attacks integrate the outputs of surrogate models with diverse architectures, which can be combined with various gradient-based attacks to improve adversarial transferability. However, previous work shows unsatisfactory attack performance when transferring across heterogeneous model archite…

• DeepTracer: Tracing Stolen Model via Deep Coupled Watermarks

  Yunfei Yang, Xiaojun Chen, Yuexin Xuan, Zhendong Zhao, Xin Zhao, He Li · 13. November 2025

  Model watermarking techniques can embed watermark information into the protected model for ownership declaration by constructing specific input-output pairs. However, existing watermarks are easily removed when facing model stealing attacks, and make it difficult for model owners to effectively veri…

• Assumed Density Filtering and Smoothing with Neural Network Surrogate Models

  Simon Kuang, Xinfan Lin · 13. November 2025

  The Kalman filter and Rauch-Tung-Striebel (RTS) smoother are optimal for state estimation in linear dynamic systems. With nonlinear systems, the challenge consists in how to propagate uncertainty through the state transitions and output function. For the case of a neural network model, we enable acc…

• GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy Algorithm

  Hanrui Wang, Ching-Chun Chang, Chun-Shien Lu, Christopher Leckie, Isao Echizen · 13. November 2025

  Deep neural networks are highly vulnerable to adversarial examples, which are inputs with small, carefully crafted perturbations that cause misclassification--making adversarial attacks a critical tool for evaluating robustness. Existing black-box methods typically entail a trade-off between precisi…

• Robust Backdoor Removal by Reconstructing Trigger-Activated Changes in Latent Representation

  Kazuki Iwahana, Yusuke Yamasaki, Akira Ito, Takayuki Miura, Toshiki Shibahara · 13. November 2025

  Backdoor attacks pose a critical threat to machine learning models, causing them to behave normally on clean data but misclassify poisoned data into a poisoned class. Existing defenses often attempt to identify and remove backdoor neurons based on Trigger-Activated Changes (TAC) which is the activat…

• Multiplicative Reweighting for Robust Neural Network Optimization

  Noga Bar, Tomer Koren, Raja Giryes · 12. November 2025

  Neural networks are widespread due to their powerful performance. Yet, they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with expert advice, where multiplicative weights (MW) updates were recently shown to be robust to moderate data corruptions in exp…

• FedShard: Federated Unlearning with Efficiency Fairness and Performance Fairness

  Siyuan Wen, Meng Zhang, Yang Yang, Ningning Ding · 12. November 2025

  To protect clients' right to be forgotten in federated learning, federated unlearning aims to remove the data contribution of leaving clients from the global learned model. While current studies mainly focused on enhancing unlearning efficiency and effectiveness, the crucial aspects of efficiency fa…

• Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networks

  Diego Garc\'ia-P\'erez, Emilio Parrado-Hern\'andez, John Shawe-Taylor · 12. November 2025

  This paper presents four theoretical contributions that improve the usability of risk certificates for neural networks based on PAC-Bayes bounds. First, two bounds on the KL divergence between Bernoulli distributions enable the derivation of the tightest explicit bounds on the true risk of classifie…