Adversarial Robustness in Machine Learning

Últimos artículos

• Model Inversion Attacks: A Survey of Approaches and Countermeasures

  Zhanke Zhou, Jianing Zhu, Fengfei Yu, Xuan Li, Xiong Peng, Tongliang Liu, Bo Han · 3 de noviembre de 2025

  The success of deep neural networks has driven numerous research studies and applications from Euclidean to non-Euclidean data. However, there are increasing concerns about privacy leakage, as these networks rely on processing private data. Recently, a new type of privacy attack, the model inversion…

• VISAT: Benchmarking Adversarial and Distribution Shift Robustness in Traffic Sign Recognition with Visual Attributes

  Simon Yu, Peilin Yu, Hongbo Zheng, Huajie Shao, Han Zhao, Lui Sha · 3 de noviembre de 2025

  We present VISAT, a novel open dataset and benchmarking suite for evaluating model robustness in the task of traffic sign recognition with the presence of visual attributes. Built upon the Mapillary Traffic Sign Dataset (MTSD), our dataset introduces two benchmarks that respectively emphasize robust…

• Fine-Grained Iterative Adversarial Attacks with Limited Computation Budget

  Zhichao Hou, Weizhi Gao, Xiaorui Liu · 3 de noviembre de 2025

  This work tackles a critical challenge in AI safety research under limited compute: given a fixed computation budget, how can one maximize the strength of iterative adversarial attacks? Coarsely reducing the number of attack iterations lowers cost but substantially weakens effectiveness. To fulfill …

• MLPerf Automotive

  Radoyeh Shojaei, Predrag Djurdjevic, Mostafa El-Khamy, James Goel, Kasper Mecklenburg, John Owens, P{\i}nar Muyan-\"Oz\c{c}elik, Tom St. John, Jinho Suh, Arjun Suresh · 3 de noviembre de 2025

  We present MLPerf Automotive, the first standardized public benchmark for evaluating Machine Learning systems that are deployed for AI acceleration in automotive systems. Developed through a collaborative partnership between MLCommons and the Autonomous Vehicle Computing Consortium, this benchmark a…

• Accelerated Rates between Stochastic and Adversarial Online Convex Optimization

  Sarah Sachs, Hedi Hadiji, Tim van Erven, Cristobal Guzman · 3 de noviembre de 2025

  Stochastic and adversarial data are two widely studied settings in online learning. But many optimization tasks are neither i.i.d. nor fully adversarial, which makes it of fundamental interest to get a better theoretical understanding of the world between these extremes. In this work we establish no…

• Fast Adversarial Training against Sparse Attacks Requires Loss Smoothing

  Xuyang Zhong, Yixiao Huang, Chen Liu · 3 de noviembre de 2025

  This paper studies fast adversarial training against sparse adversarial perturbations bounded by $l_0$ norm. We demonstrate the challenges of employing $1$-step attacks on $l_0$ bounded perturbations for fast adversarial training, including degraded performance and the occurrence of catastrophic ove…

• Conformal Object Detection by Sequential Risk Control

  L\'eo and\'eol, Luca Mossina, Adrien Mazoyer, S\'ebastien Gerchinovitz · 3 de noviembre de 2025

  Recent advances in object detectors have led to their adoption for industrial uses. However, their deployment in safety-critical applications is hindered by the inherent lack of reliability of neural networks and the complex structure of object detection models. To address these challenges, we turn …

• SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregation

  Guangzhi Su, Shuchang Huang, Yutong Ke, Zhuohang Liu, Long Qian, Kaizhu Huang · 3 de noviembre de 2025

  Multimodal large language models (MLLMs) have achieved impressive performance across diverse tasks by jointly reasoning over textual and visual inputs. Despite their success, these models remain highly vulnerable to adversarial manipulations, raising concerns about their safety and reliability in de…

• Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler

  Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, Dacheng Tao · 3 de noviembre de 2025

  Harmful fine-tuning poses critical safety risks to fine-tuning-as-a-service for large language models. Existing defense strategies preemptively build robustness via attack simulation but suffer from fundamental limitations: (i) the infeasibility of extending attack simulations beyond bounded threat …

• Angular Steering: Behavior Control via Rotation in Activation Space

  Hieu M. Vu, Tan M. Nguyen · 31 de octubre de 2025

  Controlling specific behaviors in large language models while preserving their general capabilities is a central challenge for safe and reliable artificial intelligence deployment. Current steering methods, such as vector addition and directional ablation, are constrained within a two-dimensional su…

• GSE: Group-wise Sparse and Explainable Adversarial Attacks

  Shpresim Sadiku, Moritz Wagner, Sebastian Pokutta · 31 de octubre de 2025

  Sparse adversarial attacks fool deep neural networks (DNNs) through minimal pixel perturbations, often regularized by the $\ell_0$ norm. Recent efforts have replaced this norm with a structural sparsity regularizer, such as the nuclear group norm, to craft group-wise sparse adversarial attacks. The …

• ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio-Language Models

  Weifei Jin, Yuxin Cao, Junjie Su, Minhui Xue, Jie Hao, Ke Xu, Jin Song Dong, Derui Wang · 31 de octubre de 2025

  Recent advances in Audio-Language Models (ALMs) have significantly improved multimodal understanding capabilities. However, the introduction of the audio modality also brings new and unique vulnerability vectors. Previous studies have proposed jailbreak attacks that specifically target ALMs, reveali…

• LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback

  Raffaele Mura, Giorgio Piras, Kamil\.e Luko\v{s}i\=ut\.e, Maura Pintor, Amin Karbasi, Battista Biggio · 31 de octubre de 2025

  Jailbreaks are adversarial attacks designed to bypass the built-in safety mechanisms of large language models. Automated jailbreaks typically optimize an adversarial suffix or adapt long prompt templates by forcing the model to generate the initial part of a restricted or harmful response. In this w…

• FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks

  Jialong Sun, Xinpeng Ling, Jiaxuan Zou, Jiawen Kang, Kejia Zhang · 31 de octubre de 2025

  The inherent autocorrelation of time series data presents an ongoing challenge to multivariate time series prediction. Recently, a widely adopted approach has been the incorporation of frequency domain information to assist in long-term prediction tasks. Many researchers have independently observed …

• Improving LLM Safety Alignment with Dual-Objective Optimization

  Xuandong Zhao, Will Cai, Tianneng Shi, David Huang, Licong Lin, Song Mei, Dawn Song · 31 de octubre de 2025

  Existing training-time safety alignment techniques for large language models (LLMs) remain vulnerable to jailbreak attacks. Direct preference optimization (DPO), a widely deployed alignment method, exhibits limitations in both experimental and theoretical contexts as its loss function proves subopti…

• Model Inversion Attacks Meet Cryptographic Fuzzy Extractors

  Mallika Prabhakar, Louise Xu, Prateek Saxena · 30 de octubre de 2025

  Model inversion attacks pose an open challenge to privacy-sensitive applications that use machine learning (ML) models. For example, face authentication systems use modern ML models to compute embedding vectors from face images of the enrolled users and store them. If leaked, inversion attacks can a…

• Robust LLM Unlearning with MUDMAN: Meta-Unlearning with Disruption Masking And Normalization

  Filip Sondej, Yushi Yang, Miko{\l}aj Kniejski, Marcel Windys · 30 de octubre de 2025

  Language models can retain dangerous knowledge and skills even after extensive safety fine-tuning, posing both misuse and misalignment risks. Recent studies show that even specialized unlearning methods can be easily reversed. To address this, we systematically evaluate many existing and novel compo…

• A Unified Bilevel Model for Adversarial Learning and A Case Study

  Yutong Zheng, Qingna Li · 30 de octubre de 2025

  Adversarial learning has been attracting more and more attention thanks to the fast development of machine learning and artificial intelligence. However, due to the complicated structure of most machine learning models, the mechanism of adversarial attacks is not well interpreted. How to measure the…

• Lipschitz-aware Linearity Grafting for Certified Robustness

  Yongjin Han, Suhyun Kim · 30 de octubre de 2025

  Lipschitz constant is a fundamental property in certified robustness, as smaller values imply robustness to adversarial examples when a model is confident in its prediction. However, identifying the worst-case adversarial examples is known to be an NP-complete problem. Although over-approximation me…

• On the Stability of Neural Networks in Deep Learning

  Blaise Delattre · 30 de octubre de 2025

  Deep learning has achieved remarkable success across a wide range of tasks, but its models often suffer from instability and vulnerability: small changes to the input may drastically affect predictions, while optimization can be hindered by sharp loss landscapes. This thesis addresses these issues t…

• Adversarially-Aware Architecture Design for Robust Medical AI Systems

  Alyssa Gerhart, Balaji Iyangar · 29 de octubre de 2025

  Adversarial attacks pose a severe risk to AI systems used in healthcare, capable of misleading models into dangerous misclassifications that can delay treatments or cause misdiagnoses. These attacks, often imperceptible to human perception, threaten patient safety, particularly in underserved popula…

• PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction

  Peilin He, James Joshi · 29 de octubre de 2025

  Reconstructing high-quality images from low-resolution inputs using Residual Dense Spatial Networks (RDSNs) is crucial yet challenging. It is even more challenging in centralized training where multiple collaborating parties are involved, as it poses significant privacy risks, including data leakage…

• MixAT: Combining Continuous and Discrete Adversarial Training for LLMs

  Csaba D\'ek\'any, Stefan Balauca, Robin Staab, Dimitar I. Dimitrov, Martin Vechev · 29 de octubre de 2025

  Despite recent efforts in Large Language Model (LLM) safety and alignment, current adversarial attacks on frontier LLMs can still consistently force harmful generations. Although adversarial training has been widely studied and shown to significantly improve the robustness of traditional machine lea…

• Apollo: A Posteriori Label-Only Membership Inference Attack Towards Machine Unlearning

  Liou Tang, James Joshi, Ashish Kundu · 29 de octubre de 2025

  Machine Unlearning (MU) aims to update Machine Learning (ML) models following requests to remove training samples and their influences on a trained model efficiently without retraining the original ML model from scratch. While MU itself has been employed to provide privacy protection and regulatory …

• Membership Inference Attacks for Unseen Classes

  Pratiksha Thaker, Neil Kale, Zhiwei Steven Wu, Virginia Smith · 28 de octubre de 2025

  The state-of-the-art for membership inference attacks on machine learning models is a class of attacks based on shadow models that mimic the behavior of the target model on subsets of held-out nonmember data. However, we find that this class of attacks is fundamentally limited because of a key assum…