Adversarial Robustness in Machine Learning

Latest papers

• Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Reading

  Rasul Khanbayov, Hasan Kurban · 7 August 2026

  Label-free reliability for vision-language models rests on invariance: perturb the input and a faithful reader's answer should not change. This has a known blind spot, a systematic misreading survives the perturbation and gets certified wrong, which we show is computable, not just real: an error is …

• Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots

  S. M . Bhagya P. Samarakoon, M. A. Viraj J. Muthugala, W. K. R. Sachinthana, Mohan Rajesh Elara · 7 August 2026

  Vision-Language Models (VLMs) are increasingly deployed as planners in robotic systems, where they translate natural-language commands into executable actions grounded in visual scene understanding. This tight coupling between perception and instruction-following introduces a new attack surface: adv…

• The Geometry of Flow-Matching Uncertainty: A Cost-free Uncertainty Proxy and Its Application in Flow-based VLA Failure Detection

  Ziyang Rao, Yiren Zhao, Weiyu Guo, Ben Fei, Yandong Guo, Hui Xiong · 7 August 2026

  Flow matching (FM) has become a popular action head paradigm for modern embodied models. However, as a conditional generative model, it does not explicitly expose its inherent uncertainty, producing faulty action chunks even when it misinterprets the scene or encounters out-of-distribution (OOD) inp…

• Subliminal Learning is Non-Semantic Distillation

  Ethan Hadley, Eren Gultepe · 7 August 2026

  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher. This presents challenges in ensuring A…

• AI Security Leaderboard: Methodology, Results and Minimal Standard

  Jasper Timm, Lukas Struppek, Ziwei Xu, Grace Cheong, Oscar Mata, Dan Zhao, Mick Yang, Isadora De Andrade, Xiaojun Jia, Yiming Li, Samuel Bauer, Heather McIntyre, Adam Gleave, Edward Yee, Kellin Pelrine · 7 August 2026

  The AI Security Leaderboard is an independent benchmark that ranks the safeguards of frontier AI models from least to most secure. It tests models against the FAR$.$AI Minimal Standard for Safeguards, which represents a minimum bar for security: meeting it does not guarantee a secure model, but fail…

• CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits

  Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad, Ali Abbasi, Seyedarmin Azizi · 7 August 2026

  Controlling the behavior of large language models (LLMs) remains a critical challenge for AI alignment. Existing steering methods, such as Contrastive Activation Addition (CAA), typically rely on fixed single-layer interventions derived from aggregate activation differences. These methods impose a s…

• DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models

  Zhicong Huang, Cheng Hong, Tao Wei · 6 August 2026

  Cloud-based language model services routinely process prompts containing sensitive information. Obfuscation-based defenses---including ObfusLM, SentinelLMs, TextObfuscator, and DPNR---mitigate this risk by transforming prompt representations before transmission, offering a lightweight alternative to…

• Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning

  Yuxuan Huang, Xingyu Zeng, Tianhang Zheng, Chaochao Lu · 6 August 2026

  Released aligned large language models remain vulnerable to malicious downstream finetuning. Existing defenses are largely designed for the fine-tuning-as-a-service (FTaaS) paradigm or rely on downstream users to follow additional safety procedures, and therefore do not directly address the setting …

• One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIP

  Binyan Xu, Xilin Dai, Di Tang, Kehuan Zhang · 6 August 2026

  Deep Neural Networks (DNNs) have achieved widespread success yet remain prone to adversarial attacks. Typically, such attacks either involve frequent queries to the target model or rely on surrogate models closely mirroring the target model -- often trained with subsets of the target model's trainin…

• When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning

  Yiming Chen, Kemou Li, Haiwei Wu, Jiantao Zhou · 6 August 2026

  Backdoor attacks in multimodal contrastive learning (MCL) have garnered growing attention in recent years, as many downstream tasks critically depend on pre-trained MCL models. Existing detection-based defenses predominantly rely on the CLIPScore metric, under the assumption that poisoned pairs exhi…

• Robustness Emerges Early in Training Dynamics, but Is Not Preserved

  Jiangang Yang, Wenhui Shi, Lu Hu, Jing Xing, Jian Liu · 6 August 2026

  Robustness to natural corruptions remains a fundamental challenge for deep neural networks. In this paper, we identify a robustness fading phenomenon where shallow layers spontaneously develop robust representations and flat loss landscapes in early training, yet these properties are not preserved d…

• Evading Chain-of-Thought Monitoring Through Model Poisoning

  Giorgio Severi, Shujaat Mirza, Blake Bullwinkel, Amanda Minnich · 5 August 2026

  Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks but relies on the assumption that a model's reasoning trace is informative about its actions. This work studies the limits of CoT monitoring through the lens of model poisoning. We demonstrate that backdoors…

• Inverted Detection and Control in Steering Vectors

  Max Torop, Aria Masoomi, Jennifer Dy · 5 August 2026

  Steering vectors (SVs) are widely used to influence the expression of concepts (e.g., truthfulness) in large language model outputs. A key assumption underpinning SVs is that they are linearly discriminative with respect to the concept: representations of texts that exhibit the concept are more alig…

• Temporal Leakage in LLM Backtesting: Measurement, Validation, and Adjusted Scores

  Zeyu Zhang, Bradly C. Stadie · 5 August 2026

  The standard check for contamination in LLM backtests is simple: compare scores before and after the training cutoff. We show this check is uninformative. Four flagship models fail it on questions they cannot have memorized: every scored question resolved after their cutoffs. The reason is structura…

• DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack

  Hoseong Tae, Jong-Seok Lee · 5 August 2026

  Flow-matching vision-language-action (VLA) models such as pi0 generate robot actions by integrating a learned denoising velocity field, and have been reported to resist adversarial perturbations that readily fool autoregressive VLAs. We show that this robustness is largely illusory: it stems from pr…

• Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures

  Nicola Pitzalis, Donald Shenaj, Giacomo Cignoni, Andrea Cossu, Davide Bacciu, Antonio Carta · 4 August 2026

  Parameter-Efficient Fine-tuned (PEFT) models are frequently downloaded from open repositories by practitioners. This widespread practice creates a significant attack surface, as malicious actors can publish backdoored models that induce specific behaviors in response to predefined triggers. We study…

• When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems

  Jia-Hao Xiao, Lei Feng, Min-Ling Zhang · 4 August 2026

  LLM-based multi-agent systems (MAS) extend LLM capabilities through iterative communication and shared contexts. However, this collaboration introduces a vulnerability: backdoor behavior can be activated when peer evidence reaches a hidden threshold, rather than being determined by any single messag…

• Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift

  Hanyu Su, Carlota Julbe i Juanola, Yibo Hu · 4 August 2026

  Subtype robustness asks whether a model keeps the correct coarse prediction when test examples come from fine-grained subtypes absent from training but still inside a known coarse category. Prior work studies this almost entirely through accuracy. We ask whether the model also stays calibrated. We p…

• Mitigating Backdoors via Decoy Shortcuts and Knowledge Decoupling

  Zixuan Zhu, Rui Wang, Lihua Jing, Jinwen Zhong · 4 August 2026

  Backdoor attacks pose a serious threat to deep neural networks, especially when training relies on third-party data, allowing adversaries to inject malicious behaviors through data poisoning. In this work, we reveal that backdoor behaviors tend to be absorbed by a simpler parallel branch when jointl…

• Compliance2LoRA: Personalizable On-Demand Safety Alignment on Arbitrary Policy Subsets via Hypernetwork-Generated LoRA Adapters

  Pankayaraj Pathmanathan, Furong Huang · 4 August 2026

  Post-training alignment in large reasoning models (LRMs) has significantly improved their adaptability to diverse safety compliance settings. However, as LRMs personalization for downstream users takes center stage, the demand for varying levels of policy compliance grows as different user-specific …

• Caliber: Cross-Architecture Extraction-Cost Control for Score-Returning APIs

  Chi Wang, Hanwen Wang, Yu Xia, Zihan Wang, Guangdong Bai · 4 August 2026

  We present Caliber, an output-perturbation defense against model extraction that formulates noise selection as a calibration problem: how much the defense degrades the supervision signal used to train a surrogate, and the provable per-input query cost of recovering the clean logits. To defend agains…

• Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

  Ren-Jian Wang, Ke Xue, Zeyu Qin, Ziniu Li, Sheng Tang, Hao-Tian Li, Shengcai Liu, Zhi Yu, Yuanpeng Tan, Chao Qian · 4 August 2026

  Ensuring the safety and robustness of large language models (LLMs) is a fundamental challenge and a critical prerequisite for the responsible deployment of artificial intelligence. Red-teaming, a systematic framework to identify adversarial prompts that elicit harmful responses from target LLMs, has…

• Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study

  Ramesh B. Paramkusham · 4 August 2026

  Domain adaptation of small language models (SLMs) has emerged as a practical strategy for deploying capable NLP systems in resource-constrained, high-stakes environments including healthcare, legal services, and financial analysis. While performance gains from parameter-efficient fine-tuning are wel…

• VLAGuard: A Framework for Evaluating and Mitigating Physical Attention Hijacking in Vision-Language-Action Robots within Wireless Sensor Networks

  Dongfu Yin, Jinquan Zhang · 4 August 2026

  Deploying Vision-Language-Action (VLA) robots as mobile edge nodes within wireless sensor networks (WSNs) requires robust protection against physical adversarial threats. We present VLAGuard, a framework to assess and mitigate a critical vulnerability: policy-critical action-to-vision attention hija…

• No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks

  Simiao Xie, Chuancheng Shi, Shangze Li, Wenhua Wu, Fei Shen, Ying Zhou, Zhiyong Wang, Tat-Seng Chua · 4 August 2026

  With the rapid release of open-weight large foundation models, safety threats are shifting from black-box jailbreaks to neuron-level white-box attacks that directly identify and manipulate safety-related neurons. Existing alignment methods often investigate the safety behavior on a small number of n…