Adversarial Robustness in Machine Learning

Últimos artículos

• CSULoRA: Closest Safe Update Low-Rank Adaptation

  Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh, Salima Lamsiyah · 1 de junio de 2026

  Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned models. Existing safety-preserving LoRA methods often rely on hard i…

• What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness

  Yusheng He, Jizhe Zhou, Xia Du, Zheng Lin, Jun Luo, Jiancheng Lv · 1 de junio de 2026

  Hallucination remains one of the key challenges undermining the reliability of Large Vision-Language Models (LVLMs). But what makes an LVLM hallucinate less? Many existing efforts focus on improving internal components of the model. We argue that hallucination fundamentally stems from how the model …

• From Internal Diagnosis to External Auditing: A VLM-Driven Paradigm for Data-Free Online Backdoor Defense

  Binyan Xu, Fan Yang, Xilin Dai, Di Tang, Kehuan Zhang · 1 de junio de 2026

  Deep Neural Networks remain inherently vulnerable to backdoor attacks. Traditional test-time defenses largely operate under the paradigm of internal diagnosis methods like model repairing or input robustness, yet these approaches are often fragile under advanced attacks as they remain entangled with…

• AI Loss of Control Incident Management: Response & Resilience

  Ross Gruetzemacher · 1 de junio de 2026

  Recent research demonstrating AI systems exhibiting deception and shutdown resistance suggests that AI loss of control (LOC) is an urgent policy concern , yet current literature focuses almost exclusively on alignment and prevention. To address this gap, this paper introduces a foundational framewor…

• Is the Last Layer Sufficient for Uncertainty Quantification?

  Joseph Wilson, Chris van der Heide, Liam Hodgkinson, Fred Roosta · 1 de junio de 2026

  Epistemic uncertainty quantification (UQ) for deep neural networks (DNNs) is a requirement for safe adoption of AI in mission-critical settings. Several leading methods for UQ linearize DNNs to form Bayesian Generalized Linear Models (GLMs), where epistemic uncertainty is modeled via the predictive …

• Latent Geometric Chords for Query-Efficient Decision-Based Adversarial Attacks

  Ei Hmue Khine, Yao Li, Jiebao Sun, Shengzhu Shi, Zhichang Guo, Boying Wu · 1 de junio de 2026

  While decision-based black-box adversarial attacks present a severe security threat, current methodologies suffer from fundamental limitations. Pixel-wise attacks frequently introduce unnatural, high-frequency visual artifacts, while latent-space frameworks are confined by the limited search space o…

• Bounded Behavioral Indistinguishability for Black-Box LLM Distillation

  Munawar Hasan · 1 de junio de 2026

  Black-box LLM distillation is usually evaluated as an output-matching problem: a student is considered successful when its responses are semantically similar to, or task-consistent with, those of a teacher. However, output similarity does not imply that the student is behaviorally indistinguishable …

• Geometry-based Schr\"odinger Bridges for Trustworthy Multimodal Fusion

  Jiayu Xiong, Jing Wang, Qi Zhang, Wanlong Wang, Jun Xue · 1 de junio de 2026

  Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a…

• Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles

  Mehmet Ozgur Turkoglu, Dominik J. M\"uhlematter, Alexander Becker, Konrad Schindler, Helge Aasen · 1 de junio de 2026

  Foundation models have become a dominant paradigm in machine learning, achieving remarkable performance across diverse tasks through large-scale pretraining. However, they often yield overconfident, uncalibrated predictions. The standard approach to quantifying epistemic uncertainty are ensembles of…

• Non-Parametric Probabilistic Robustness: A Conservative Risk Estimator under Unknown Perturbation Distributions

  Zheng Wang, Yi Zhang, Siddartha Khastgir, Carsten Maple, Xingyu Zhao · 1 de junio de 2026

  Deep learning (DL) models, despite their remarkable success, remain vulnerable to small input perturbations that can cause erroneous outputs, motivating the recent proposal of probabilistic robustness (PR) as a complementary alternative to adversarial robustness (AR). However, existing PR formulatio…

• TASER: Task-Aware Stein Regularisation for Geometry-Driven Robustness

  Micha{\l} Kozyra, Gesine Reinert · 1 de junio de 2026

  Modern deep networks remain fragile under distribution shift and adversarial perturbations, often due to excessive or poorly structured input sensitivity. We introduce TASER (Task-Aware Stein Regularisation), a training-time regularisation framework derived from Langevin Stein operators. By penalisi…

• Token Inflation: How Dishonest Providers Can Overcharge for Large Language Model Usage

  Shahinul Hoque, Jinghuai Zhang, Jinyuan Sun, Fnu Suya · 29 de mayo de 2026

  Per-token billing is now the standard pricing model for commercial large language models (LLMs), so the honesty of reported token counts directly affects what users pay. We show that this kind of billing is hard to audit by design: providers hide the model, the tokenizer, and the execution to protec…

• Improving Adversarial Robustness of Attribution via Implicit Regularization

  Amir Mehrpanah, Matteo Gamba, Hossein Azizpour · 29 de mayo de 2026

  The adversarial robustness of attributions is a fundamental requirement for reliable explainability in deep learning, yet existing approaches typically rely on computationally expensive explicit regularization. In this work, we show that attribution robustness can arise implicitly from the learning …

• Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation

  Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu, You-Hsuan Chang, Yun-Nung Chen · 29 de mayo de 2026

  Large Audio Language Models (LALMs) expand jailbreak risks from token-level prompting to the full speech perception-to-reasoning pipeline, where unsafe behavior can be induced through semantics, acoustic style, signal artifacts, or internal representations. Existing work studies these risks under he…

• Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interaction

  Hongtao Wang, Se Yang, Yu Chen, Puzhuo Liu · 29 de mayo de 2026

  Large language model (LLM) agents increasingly leverage long term memory to support persistent and autonomous task execution. However, this capability also introduces a new attack surface: memory poisoning, where adversaries can inject malicious information to influence future behavior. Existing mem…

• Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering

  Soumyadeep Jana, Pulkit Mittal, Sanasam Ranbir Singh · 29 de mayo de 2026

  Large vision-language models (LVLMs) often hallucinate objects that are not present in the input image, largely because visual grounding weakens as decoding progresses. Existing inference-time mitigation methods modify logits or hidden states throughout generation, but they suffer from three key lim…

• Neural Network Verification using Partial Multi-Neuron Relaxation

  Ido Shmuel, Guy Katz · 29 de mayo de 2026

  The increasing integration of deep neural networks in critical systems has spawned a theoretical and practical interest in formally guaranteeing safety properties about their behavior. To achieve this, contemporary verification algorithms rely on computing linear relaxations for a network's non-line…

• MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization

  Anisha Saha, Varsha Suresh, Teodora Kamova, Sophia Wiedmann, Timothy Hospedales, Vera Demberg · 29 de mayo de 2026

  Understanding how harm emerges from interaction between otherwise benign image-text pairs requires intent-aware cross-modal reasoning beyond surface-level features. Existing vision-language models (VLMs) excel at literal reasoning over perceptual cues but often fail to derive harmful semantics that …

• Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization

  Zhihao Liu, Yifan Wu, Jian Lou, Di Wang, Yuxi Zhou, Yuke Hu · 29 de mayo de 2026

  Safety alignment for large language models (LLMs) aims to reduce harmful or unsafe behavior while preserving general utility. However, recent findings reveal that alignment effects can be fragile: lightweight post-alignment manipulations, such as parameter noise, activation noise, or quantization, c…

• Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures

  Junyoung Park, Sunghwan Park, Seongyong Ju, Jaewoo Lee · 29 de mayo de 2026

  Attack Success Rate (ASR) evaluates each jailbreak with a single yes/no label at the end of generation, telling us whether a failure happened but not how it unfolded. Two attacks that produce equally harmful outputs may have followed completely different paths, and ASR cannot tell them apart. We mak…

• Harnessing non-adversarial robustness in large language models

  Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin, Oleg Somov, Mikhail Seleznyov, Alexander Panchenko, Ivan Oseledets, Elena Tutubalina, Ivan Y. Tyukin · 29 de mayo de 2026

  The work presents an approach for addressing the challenge of robustness in Large Language Models (LLMs) to alterations and potential errors caused by semantically similar but textually different prompts. Recent works have shown that these kinds of prompt variations can significantly impact the perf…

• Quantum-Enhanced Adversarial Robustness in Artificial Intelligence

  Jaydip Sen · 29 de mayo de 2026

  Artificial Intelligence has achieved remarkable success across diverse application domains. However, its vulnerability to adversarial attacks poses significant challenges to reliability, security, and trustworthiness. Adversarial machine learning demonstrates that even highly accurate models can be …

• Convex Basins in Single-Index Model Loss Landscapes: Applications to Robust Recovery under Strong Adversarial Corruption

  Santanu Das, Sagnik Chatterjee, Jatin Batra · 29 de mayo de 2026

  We study the problem of robustly learning Gaussian Single Index Models (SIMs) in the presence of heavy-tailed noise and a constant fraction of adversarially corrupted covariates and responses. Prior work on robust recovery has considered settings such as linear regression (Pensia et al., JASA 2024),…

• A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning

  Ding Chen, Xinwen Cheng, Xuyang Zhong, Xinping Chen, Xiaolin Huang, Chen Liu · 29 de mayo de 2026

  While Membership Inference Attacks (MIAs) are the prevailing method for identifying training data, their application has expanded into privacy auditing and machine unlearning. Nevertheless, the field lacks a systematic framework for evaluating how different contexts affect MIA efficacy. Without such…

• TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models

  Yang Dai, Oubo Ma, Longfei Zhang, Xingxing Liang, Xiaochun Cao, Shouling Ji, Jiaheng Zhang, Jincai Huang, Li Shen · 29 de mayo de 2026

  Recent advances in Trajectory Optimization (TO) models have achieved remarkable success in offline reinforcement learning. However, their vulnerabilities against backdoor attacks are poorly understood. We find that existing backdoor attacks in reinforcement learning are based on reward manipulation,…