Adversarial Robustness in Machine Learning

Últimos artículos

• Precise Verification of Transformers through ReLU-Catalyzed Abstraction Refinement

  Hengjie Liu, Zhenya Zhang, Jianjun Zhao · 15 de mayo de 2026

  Formal verification of transformers has become increasingly important due to their widespread deployment in safety-critical applications. Compared to classic neural networks, the inferences of transformers involve highly complex computations, such as dot products in self-attention layers, rendering …

• NodeSynth: Socially Aligned Synthetic Data for AI Evaluation

  Qazi Mamunur Rashid, Xuan Yang, Zhengzhe Yang, Yanzhou Pan, Erin van Liemt, Darlene Neal, Kshitij Pancholi, Jamila Smith-Loud · 15 de mayo de 2026

  Recent advancements in generative AI facilitate large-scale synthetic data generation for model evaluation. However, without targeted approaches, these datasets often lack the sociotechnical nuance required for sensitive domains. We introduce NodeSynth, an evidence-grounded methodology that generate…

• Selective Safety Steering via Value-Filtered Decoding

  Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh, Yarin Gal, Yaniv Romano · 15 de mayo de 2026

  While large language models (LLMs) are trained to align with human values, their generations may still violate safety constraints. A growing line of work addresses this problem by modifying the model's sampling policy at decoding time using a safety reward. However, existing decoding-time steering m…

• Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion

  Chenyi Wang, Ruoyu Song, Raymond Muller, Jean-Philippe Monteuuis, Jonathan Petit, Z. Berkay Celik, Ryan Gerdes, Ming F. Li · 15 de mayo de 2026

  Autonomous vehicles depend on online HD map construction to perceive lane boundaries, dividers, and pedestrian crossings -- safety-critical road elements that directly govern motion planning. While existing pixel perturbation attacks can disrupt the mapping, they can be neutralized by standard adver…

• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution

  Saisab Sadhu, Pratinav Seth, Vinay Kumar Sankarapu · 15 de mayo de 2026

  Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model being quantized first. Recent work has shown that 4-bit post-training quantization can reverse machine unlearning; we show this is not a tuning artefact…

• MahaVar: OOD Detection via Class-wise Mahalanobis Distance Variance under Neural Collapse

  Donghwan Kim, Hyunsoo Yoon · 15 de mayo de 2026

  Out-of-distribution (OOD) detection is a critical component for ensuring the reliability of deep neural networks in safety-critical applications. In this work, we present a key empirical observation: for in-distribution (ID) samples, class-wise Mahalanobis distances exhibit a pronounced sharp minimu…

• Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment

  Yuchen Sun, Pei Fu, Shaojie Zhang, Anan Du, Xiuwen Xi, Ruoceng Zhang, Zhenbo Luo, Jian Luan, Chongyang Zhang · 15 de mayo de 2026

  Test-Time Scaling (TTS), which samples multiple candidate actions and ranks them via a Critic Model, has emerged as a promising paradigm for generalist GUI agents. Its efficacy thus hinges on the critic's fine-grained ranking ability. However, existing GUI critic models uniformly adopt binary classi…

• Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning

  Oubo Ma, Ruixiao Lin, Yang Dai, Jiahao Chen, Chunyi Zhou, Linkang Du, Shouling Ji · 15 de mayo de 2026

  Extensive research has highlighted the severe threats posed by backdoor attacks to deep reinforcement learning (DRL). However, prior studies primarily focus on vanilla scenarios, while plasticity interventions have emerged as indispensable built-in components of modern DRL agents. Despite their effe…

• Hidden State Poisoning Attacks against Mamba-based Language Models

  Alexandre Le Mercier, Chris Develder, Thomas Demeester · 15 de mayo de 2026

  State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect…

• To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model

  Chengshuai Zhao, Zhen Tan, Dawei Li, Zhiyuan Yu, Huan Liu · 15 de mayo de 2026

  The rapid advancement of Large Vision-Language Models (LVLMs) is increasingly accompanied by unauthorized scraping and training on multimodal web data, posing severe copyright and privacy risks to data owners. Existing countermeasures, such as machine unlearning and watermarks, are inherent post-hoc…

• Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding

  Shuoyang Sun, Chang Da, Hao Fang, Kuofeng Gao, Xinhao Zhong, Yi Sun, Fan Mo, Shu-Tao Xia, Bin Chen · 15 de mayo de 2026

  Speculative decoding has become a widely adopted technique for accelerating large language model (LLM) inference by drafting multiple candidate tokens and verifying them with a target model in parallel. Its efficiency, however, critically depends on the average accepted length $\tau$, i.e., how many…

• XAI and Statistical Analysis for Reliable Intrusion Detection in the UAVIDS-2025 Dataset: From Tree to Hybrid and Tabular DNN Ensembles

  Iakovos-Christos Zarkadis, Christos Douligeris · 15 de mayo de 2026

  During the last few years, the term Mechanistic Interpretability, a specific area, under the umbrella of explainable artificial intelligence (XAI), has been introduced, to explain the decisions made by complex machine learning (ML) models in critical systems like UAV intrusion detection systems (UAV…

• Fair and Calibrated Toxicity Detection with Robust Training and Abstention

  Mokshit Surana · 15 de mayo de 2026

  Fairness in toxicity classification involves three integrated axes: ranking, calibration, and abstention. Training-time interventions and post-hoc safety mechanisms cannot be evaluated independently because the former determines the efficacy of the latter. We compare Empirical Risk Minimization (ERM…

• Fast Adversarial Attacks with Gradient Prediction

  Kamil Ciosek, Aleksandr V. Petrov, Nicol\`o Felicioni, Konstantina Palla · 15 de mayo de 2026

  Generating adversarial examples at scale is a core primitive for robustness evaluation, adversarial training, and red-teaming, yet even "fast" attacks such as FGSM remain throughput-limited by the cost of a backward pass. We introduce a family of attacks that eliminates the backward pass by predicti…

• Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study

  Mokshit Surana, Archit Rathod, Akshaj Satishkumar · 15 de mayo de 2026

  Large Language Models (LLMs), when trained on web-scale corpora, inherently absorb toxic patterns from their training data. This leads to ``toxic degeneration'' where even innocuous prompts can trigger harmful outputs. This phenomenon poses significant risks for real-world deployments. Thus, necessi…

• ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety

  Michael S. Lee, Yash Maurya, Drew Rein, Bert Herring, Jonathan Nguyen, Kyungho Song, Udari Madhushani Sehwag, Jiyeon Cho, Kaustubh Deshpande, Yeongkyun Jang, Jiyeon Joo, Minn Seok Choi, Evi Fuelle, Christina Q Knight, Joseph Brandifino, Max Fenkell · 15 de mayo de 2026

  Safety evaluations for large language models (LLMs) increasingly target high-stakes National Security and Public Safety (NSPS) risks, yet multilingual safety is typically assessed through translation-only benchmarks that preserve the underlying scenario, and empirical evidence of how language and ge…

• Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

  Blaise Delattre, Hengyu Wu, Paul Caillon, Wei Yang Bryan Lim, Yang Cao · 14 de mayo de 2026

  Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adv…

• Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications

  Maxwell Standen, Junae Kim, Claudia Szabo · 14 de mayo de 2026

  Multi-agent systems rely on communication for information sharing and action coordination, which exposes a vulnerability to attacks. We investigate single-victim communication perturbation attacks against Multi-Agent Reinforcement Learning-trained systems and propose methods that use gradient inform…

• Taming the Long Tail: Rebalancing Adversarial Training via Adaptive Perturbation

  Lilin Zhang, Yimo Guo, Yue Li, Jiancheng Shi, Xianggen Liu · 14 de mayo de 2026

  Deep neural networks are highly vulnerable to adversarial examples, i.e.,small perturbations that can significantly degrade model performance. While adversarial training has become the primary defense strategy, most studies focus on balanced datasets, overlooking the challenges posed by real-world l…

• Understanding Generalization through Decision Pattern Shift

  Huiqi Deng, Yibo Li, Quanshi Zhang, Peng Zhang, Hongbin Pei, Xia Hu · 14 de mayo de 2026

  Understanding why deep neural networks (DNNs) fail to generalize to unseen samples remains a long-standing challenge. Existing studies mainly examine changes in externally observable factors such as data, representations, or outputs, yet offer limited insight into how a model's internal decision mec…

• SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions

  Thushari Hapuarachchi, Kaiqi Xiong · 14 de mayo de 2026

  There is recently a serious issue that Deep Neural Networks (DNNs) training uses more and more unauthorized data. A clean-label generalization attack, one type of data poisoning attacks, has been suggested to address this issue. The Neural Tangent Generalization Attack (NTGA) is considered as the fi…

• Shortcut Mitigation via Spurious-Positive Samples

  Phuong Quynh Le, J\"org Schl\"otterer, Sari Sadiya, Gemma Roig, Christin Seifert · 14 de mayo de 2026

  Shortcut mitigation strategies commonly rely on training data annotations, group-balanced held-out data or the presence of all groups, i.e., all combinations of (spurious) attributes and classes, in the training data. However, these requirements are rarely met in practice. We instead propose a metho…

• Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing

  Danial Samadi Vahdati, Tai Duc Nguyen, Ekta Prashnani, Koki Nagano, David Luebke, Orazio Gallo, Matthew Stamm · 14 de mayo de 2026

  AI-based talking-head videoconferencing systems reduce bandwidth by sending a compact pose-expression latent and re-synthesizing RGB at the receiver, but this latent can be puppeteered, letting an attacker hijack a victim's likeness in real time. Because every frame is synthetic, deepfake and synthe…

• Sockpuppetting: Jailbreaking LLMs by Combining Prefilling with Optimization

  Asen Dotsinski, Panagiotis Eustratiadis · 14 de mayo de 2026

  Prefill attacks are an effective and low-cost jailbreaking method, as they directly insert an acceptance sequence (e.g., "Sure, here is how to...") at the start of an LLM's output and lead the model to continue the response. We make two contributions to this prior work. First, we show that an unsoph…

• Where Do Reasoning Models Refuse?

  Kureha Yamaguchi, Benjamin Etheridge, Andy Arditi · 14 de mayo de 2026

  Chat models without chain-of-thought (CoT) reasoning must decide whether to refuse a harmful request before generating their first response token. Reasoning models, by contrast, produce extended chains of thought before their final output, raising a natural question: where in this process does the d…