Adversarial Robustness in Machine Learning

Neueste Paper

• Halt Fast! Early Stopping for Certified Robustness

  Andrew C. Cullen, Paul Montague, Benjamin I. P. Rubinstein · 29. Juni 2026

  Randomized Smoothing (RS) provides rigorous robustness guarantees for neural networks without architectural constraints, yet its adoption is limited by extreme computational costs. Standard RS requires tens of thousands of model evaluations per input and forces practitioners to commit to fixed sampl…

• Adversarial Contamination Meets Hard Thresholding: An Iterative Algorithm with Signal Adaptivity and Minimax Optimality

  Shixiang Liu, Hanming Yang · 29. Juni 2026

  Pervasive data contamination -- stemming from measurement errors, outliers, or adversarial corruption -- has motivated the development of robust statistical methods. In this context, we propose a two-stage Adversarial Contamination-resistant Iterative Hard Thresholding (AC-IHT) algorithm for high-di…

• Improving Adversarial Robustness via Activation Amplification and Attenuation

  Ta\"iga Gon\c{c}alves, Yongsong Huang, Tomo Miyazaki, Shinichiro Omachi · 29. Juni 2026

  The existence of adversarial attacks is often attributed to the presence of non-robust features in neural networks. While prior defenses reduce their impact via pruning, masking, or feature recalibration, we instead propose to jointly learn to amplify and attenuate these signals through a simple act…

• USAD: Uncertainty-aware Statistical Adversarial Detection

  Zhijian Zhou, Xunye Tian, Jiacheng Zhang, Zesheng Ye, Yiyi Guo, Donghao Zhang, Liuhua Peng, Feng Liu · 29. Juni 2026

  Statistical adversarial detection (SAD) treats detection as a two-sample test. Given a reference set of clean examples (CEs) and a batch of queries, potentially containing an unknown mixture of CEs and adversarial examples (AEs), SAD decides whether the query distribution drifts away from the CE dis…

• From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models

  Ponhvoan Srey, Xiaobao Wu, Cong-Duy Nguyen, Quang Minh Nguyen, Duc Anh Vu, Anh Tuan Luu · 29. Juni 2026

  Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting,…

• Room for Error: Large-Scale Simulation of Over-the-Air Acoustic Attacks

  Andrew C. Cullen, Neil Marchant, Jiani Xie, Paul Montague, Benjamin I. P. Rubinstein · 29. Juni 2026

  While voice control is rapidly becoming a ubiquitous vector of human-AI communication, the risks facing these systems remain poorly understood. This is, in part, a product of the difficulties in scaling strictly digital adversarial workflows to the physical world. These scale barriers have led the c…

• Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding

  Liu Yu, Can Chen, Ping Kuang, Zhikun Feng, Fan Zhou, Gillian Dobbie · 29. Juni 2026

  Large Vision-Language Models (LVLMs) exhibit sophisticated reasoning but remain susceptible to object hallucination. Deviating from the prevailing attention intensity assumption, we reveal a deeper dynamic structural misalignment: hallucination is triggered at decision-critical steps where specific …

• Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models

  Yanchen Yin, Dongqi Han, Linghui Li · 29. Juni 2026

  Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads. We identify two functionally differentiated types: Adversarially…

• Delayed Verification Destabilizes Multi-Agent LLM Belief: Instability Thresholds and Optimal Corrector Placement

  Igor Itkin · 29. Juni 2026

  Multi-agent large language model (LLM) systems often rely on verifier and critic agents to suppress hallucinations, but verification is delayed. During this delay, false claims can propagate through the agent network. We model this process as delayed consensus on a graph with grounded corrector node…

• MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG

  Inderjeet Singh, Andr\'es Murillo, Motoyoshi Sekiya, Yuki Unno, Junichi Suga · 26. Juni 2026

  Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation. Existing red-teaming approaches are typically surface-specific and often recyc…

• At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization

  Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, Danilo Bzdok · 26. Juni 2026

  Pre-trained transformers have demonstrated remarkable generalization abilities, at times extending beyond the scope of their training data. Yet, real-world deployments often face unexpected or adversarial data that diverges from training data distributions. Without explicit mechanisms for handling s…

• The Role of Input Dimensionality in the Emergence and Targeted Control of Adversarial Examples

  Nasrin Malekzadeh Goradel, Niccolo Pancino, Yaser Gholizade Atani, Benedetta Tondi, Giovanni Bellettini, Mauro Barni · 26. Juni 2026

  Several theoretical works have tried to explain the adversarial vulnerability of deep neural networks through properties of high-dimensional geometry. However, the assumptions underlying these works are rarely examined empirically, and systematic evidence remains limited. In this work, we present a …

• Adversarial Robustness of AI-Generated Image Detectors in the Real World

  Sina Mavali, Jonas Ricker, David Pape, Asja Fischer, Lea Sch\"onherr · 26. Juni 2026

  The rapid advancement of Generative Artificial Intelligence (GenAI) capabilities is accompanied by a concerning rise in its misuse. In particular the generation of credible misinformation in the form of images poses a significant threat to the public trust in democratic processes. Consequently, ther…

• Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations

  Hiroki Tamba · 26. Juni 2026

  LLM-as-judge ("grader") components are now standard in evaluation harnesses, including safety evaluations where a pass/fail verdict may gate downstream deployment decisions. A widespread assumption is that setting the grader's sampling temperature to 0 makes grading deterministic. We test this assum…

• TEMPO-Diffusion: Temporally Exposed Malicious Poisoning of Diffusion Models

  William Aiken, Paula Branco, Guy-Vincent Jourdan, Iosif-Viorel Onut · 26. Juni 2026

  Noise-based backdoor attacks on diffusion models typically rely on input-time trigger injection, untargeted activation, and out-of-distribution target generation. Such assumptions reduce both the stealthiness and the practical relevance of these attacks. In this work, we present TEMPO-Diffusion, a t…

• Decision-Aligned Evaluation of Uncertainty Quantification

  Annika Schneider, Tommy Rochussen, Joshua Stiller, Vincent Fortuin · 26. Juni 2026

  Uncertainty estimates in machine learning are typically evaluated using generic metrics such as the negative log-likelihood and expected calibration error, yet good performance on such metrics does not necessarily imply high utility in downstream decisions. We introduce decision-alignment, a criteri…

• HauntAttack: When Attack Follows Reasoning as a Shadow

  Jingyuan Ma, Rui Li, Zheng Li, Junfeng Liu, Heming Xia, Lei Sha, Zhifang Sui · 26. Juni 2026

  Emerging Large Reasoning Models (LRMs) consistently excel in mathematical and reasoning tasks, showcasing remarkable capabilities. However, the enhancement of reasoning abilities and the exposure of internal reasoning processes introduce new safety vulnerabilities. A critical question arises: when r…

• Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

  Kai Tang, Jinhao You, Yichen Guo, Yiding Sun, Dongxu Zhang, Wenya Wang, Hanze Li, Tao Luo, Renyuan Li, Xiande Huang · 26. Juni 2026

  Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucinations, where generated content is inconsistent with the input image. Existing training-free hallucination mitigation methods often suffer from unstable performance and high sensitivity to…

• The Red Queen G\"odel Machine: Co-Evolving Agents and Their Evaluators

  Alex Iacob, Andrej Jovanovi\'c, William F. Shen, Daniel Burkhardt, Meghdad Kurmanji, Nurbek Tastan, Lorenzo Sani, Niccol\`o Alberto Elia Venanzi, Ambroise Odonnat, Zeyu Cao, Bill Marino, Xinchi Qiu, Nicholas D. Lane · 26. Juni 2026

  Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improve…

• Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation

  Ryan Fetterman · 26. Juni 2026

  LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data. We show that this can miss vulnerabilities introduced by fine-tuning itself: models can learn token-level indicator semantics that preserve canonical accuracy whi…

• Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text

  Manjinder Singh, Alexander E. I. Brownlee, Mohamed Elawady · 26. Juni 2026

  Deep learning models have achieved impressive performance across various fields but remain vulnerable to adversarial inputs, particularly in NLP, where such attacks can have significant real-world consequences. Adversarial attacks often involve small, semantically similar token replacements to fool …

• RAS: Measuring LLM Safety Through Refusal Alignment

  Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, Wei-Bin Lee · 25. Juni 2026

  Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy. Although useful, output-level evaluation is expensive, sensitive to judge choice, and easily tied to fixed question b…

• Black-Box Assisted Regression: Phase Transitions and Minimax Optimality

  Yan Zhou · 25. Juni 2026

  Foundation models are often used as fixed black-box predictors for downstream tasks with limited labeled data, but their predictions may be biased and unsafe to trust blindly. We study this setting through black-box assisted nonparametric regression: a learner observes labeled samples and can query …

• Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion

  Sahil Kadadekar · 25. Juni 2026

  Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs? We answer with Typical-Acceptance Invariance Screen (TAIS), a behavioral-…

• Certification of Machine Learning Models via Directional Sharpness

  Gefei Tan, Adria Gascon, Sarah Meiklejohn, Mariana Raykova · 25. Juni 2026

  In machine learning, model certification has been identified as an important method for gaining assurance about a model's trustworthiness and quality. A model's quality is largely determined by its ability to generalize, i.e., to perform well on data beyond what it was trained on. It is not possible…