Adversarial Robustness in Machine Learning

Neueste Paper

• MIND: Monge Inception Distance for Generative Models Evaluation

  Quentin Berthet, Yu-Han Wu, Clement Crepy, Romuald Elie, Klaus Greff, Michael Eli Sander · 11. Mai 2026

  We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fr\'echet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal t…

• Disagreement-Regularized Importance Sampling for Adversarial Label Corruption

  Csongor Horv\'ath, Ida-Maria Sintorn, Prashant Singh · 11. Mai 2026

  Standard Importance Sampling (IS) collapses under label corruption because high-norm examples, prioritized for variance reduction, are often adversarial outliers. We formalize this misalignment using an $\varepsilon$-contamination model and propose Disagreement-Regularized Importance Sampling (DR-IS…

• Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs

  Hao Wang, Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh, Daisuke Kawahara · 11. Mai 2026

  Vision-language models (VLMs) have advanced rapidly and are increasingly deployed in real-world applications, especially with the rise of agent-based systems. However, their safety has received relatively limited attention. Even the latest proprietary and open-weight VLMs remain highly vulnerable to…

• Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottlenecks

  Guoxin Lu, Letian Sha, Qing Wang, Peijie Sun, Hao Zhou, Hua Dai, Fu Xiao · 11. Mai 2026

  The safety alignment of Large Language Models (LLMs) remains vulnerable to Harmful Fine-tuning (HFT). While existing defenses impose constraints on parameters, gradients, or internal representations, we observe that they can be effectively circumvented under persistent HFT. Our analysis traces this …

• Dataset Watermarking for Closed LLMs with Provable Detection

  Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang Wang · 11. Mai 2026

  Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may have been trained on proprietary datasets or the same benchmarks used for evaluation. This motivates the need for dataset watermarking: designing datas…

• Why Does Agentic Safety Fail to Generalize Across Tasks?

  Yonatan Slutzky, Yotam Alexander, Tomer Slor, Yoav Nagel, Nadav Cohen · 11. Mai 2026

  AI agents are increasingly deployed in multi-task settings, where the task to perform is specified at test time, and the agent must generalize to unseen tasks. A major concern in such settings is safety: often, an agent must not only execute unseen tasks, but do so while avoiding risks and handling …

• A Generalized Singular Value Theory for Neural Networks

  Brian Charles Brown, Robert Bridges, David Grimsman, Mauricio Munoz, Sean Warnick · 11. Mai 2026

  Building on the abstract Generalized Singular Value Decomposition (GSVD) theory of Brown et al. [2025], we prove that most modern neural architectures admit a generalized SVD representation in which they are left-invertible before a final linear layer, with no change in input-output behavior. Furthe…

• OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing

  Jianming Chen, Yawen Wang, Junjie Wang, Zhe Liu, Qing Wang, Fanjiang Xu · 11. Mai 2026

  Tool-calling text-to-image (T2I) agents can plan and execute multi-step tool chains to accomplish complex generation and editing queries. However, this capability introduces a new safety attack surface: harmful outputs may arise from tool orchestration, where individually benign steps combine into u…

• CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios

  Taein Lim, Seongyong Ju, Munhyeok Kim, Hyunjun Kim, Hoki Kim · 11. Mai 2026

  Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity. In this paper, we reveal an interesting phenomenon: different agents exhibit distinct attack patterns. Specifically, each agent exhibits an attack-selection bias, disproportionately concentrating …

• InvThink: Premortem Reasoning for Safer Language Models

  Yubin Kim, Taehan Kim, Eugene Park, Chunjong Park, Cynthia Breazeal, Daniel McDuff, Hae Won Park · 11. Mai 2026

  We present InvThink, a training and prompting framework that requires the model to enumerate, analyze, and constrain potential failures before generating its final response. Unlike existing safety alignment methods that optimize only for safe final responses, InvThink structures generation into thre…

• EviDep: Trustworthy Multimodal Depression Estimation via Disentangled Evidential Learning

  Fangyuan Liu, Sirui Zhao, Zeyu Zhang, Jinyang Huang, Feng-Qi Cui, Bin Luo, Meng Li, Tong Xu, Enhong Chen · 11. Mai 2026

  Automated multimodal depression estimation in unconstrained environments is inherently challenged by naturalistic noise and complex behavioral variability. Prevailing deterministic methods, however, produce uncalibrated point estimates without quantifying predictive uncertainty, exposing decision-ma…

• VNN-LIB 2.0: Rigorous Foundations for Neural Network Verification

  Ann Roy, Allen Antony, Andrea Gimelli, Matthew L. Daggitt · 11. Mai 2026

  Neural network verification is an active and rapidly maturing research area, with a growing ecosystem of solvers and tools. The VNN-LIB standard was introduced to support interoperability in this ecosystem, but Version~1.0 has several serious short-comings as a formal foundation: it lacks a precise …

• From Clouds to Hallucinations: Atmospheric Retrieval Hijacking in Remote Sensing Vision-Language RAG

  Jiaju Han, Chao Li, Chengyin Hu, Qike Zhang, Xuemeng Sun, Xin Wang, Fengyu Zhang, Xiang Chen, Yiwei Wei, Jiahuan Long, Jiujiang Guo · 11. Mai 2026

  Multimodal RAG systems increasingly rely on vision-language retrievers to ground visual queries in external textual evidence. Existing adversarial studies on RAG mainly manipulate the retrieval corpus or memory, while attacks on vision-language and remote sensing models typically target end-task pre…

• Streaming Adversarial Robustness in Fuzzy ARTMAP: Mechanism-Aligned Evaluation, Progressive Training, and Interpretable Diagnostics

  Shane Cairns, Leonardo Enzo Brito da Silva, Sasha Petrenko, Donald C. Wunsch II, Jian Liu · 11. Mai 2026

  Adversarial robustness has been studied extensively for offline deep networks, but less is known about strict single-pass streaming neural learners. This paper studies adversarial robustness in Fuzzy ARTMAP, an Adaptive Resonance Theory architecture based on category competition, complement coding, …

• Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs

  Alexandre Cristov\~ao Maiorano · 11. Mai 2026

  Educational LLM tutors face a core AI alignment challenge: they must follow user intent while preserving pedagogical constraints and safety policies. We present an evaluation methodology for prompt-injection defenses in this setting, showing that guardrail design entails explicit trade-offs among ad…

• SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

  Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei, Mohammad Rostami, Jesse Thomason, Robin Jia · 11. Mai 2026

  Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining. Most existing methods require a retain set of curated examples to prevent catastrophic degradation of gener…

• Activation Differences Reveal Backdoors: A Comparison of SAE Architectures

  Sachin Kumar · 11. Mai 2026

  Backdoor attacks on language models pose a significant threat to AI safety, where models behave normally on most inputs but exhibit harmful behavior when triggered by specific patterns. Detecting such backdoors through mechanistic interpretability remains an open challenge. We investigate two sparse…

• Exposing and Mitigating Temporal Attack in Deepfake Video Detection

  Zheyuan Gu, Minghao Shao, Zhen Wang, Yusong Wang, Mingkun Xu, Shijie Zhang, Hao Jiang · 11. Mai 2026

  While spatiotemporal deepfake detectors achieve high AUC, our experiments reveal their susceptibility to evasion attacks. These models tend to overfit on fragile temporal spectrum cues, rather than learning robust semantic causality. To mitigate this vulnerability, we propose SpInShield, a temporal …

• Vaporizer: Breaking Watermarking Schemes for Large Language Model Outputs

  Jonathan Hong Jin Ng, Anh Tu Ngo, Anupam Chattopadhyay · 11. Mai 2026

  In this paper, we investigate the recent state-of-the-art schemes for watermarking large language models (LLMs) outputs. These techniques are claimed to be robust, scalable and production-grade, aimed at promoting responsible usage of LLMs. We analyse the effectiveness of these watermarking techniqu…

• Flexible Routing via Uncertainty Decomposition

  Charlotte Peale, Siddartha Devic, Parikshit Gopalan, Udi Wieder, Aravind Gollakota · 11. Mai 2026

  A key strategy for balancing performance and cost in modern machine learning systems is to dynamically route queries to either a low-cost model or a more expensive oracle (such as a large pretrained model or human expert), an approach known as model routing. In this work we present a new uncertainty…

• HBEE: Human Behavioral Entropy Engine -- Pre-Registered Multi-Agent LLM Simulation of Peer-Suspicion-Based Detection Inversion

  Vickson Ferrel · 11. Mai 2026

  Insider threat detection assumes that an adaptive insider leaves behavioral residue distinguishing them from legitimate users. We test this assumption against an LLM-driven adaptive insider in a controlled multi-agent simulator. Our pre-registered five-condition study isolates defender mode (cascade…

• MELD: Multi-Task Equilibrated Learning Detector for AI-Generated Text

  Chenjun Li, Cheng Wan, Johannes C. Paetzold · 11. Mai 2026

  Large language models are now embedded in everyday writing workflows, making reliable AI-generated text detection important for academic integrity, content moderation, and provenance tracking. In practice, however, a detector must do more than achieve high aggregate AUROC on clean, in-distribution h…

• A Systematic Investigation of The RL-Jailbreaker in LLMs

  Montaser Mohammedalamen, Kevin Roice, Reginald McLean, Alyssa Lefaivre \v{S}kopac · 11. Mai 2026

  The evolution of generative models from next-token predictors to autonomous engines of complex systems necessitates rigorous safety hardening. Adversarial jailbreaking, the strategic manipulation of models to elicit harmful output, remains a primary threat to safe deployment. While Reinforcement Lea…

• Hallucination Detection via Activations of Open-Weight Proxy Analyzers

  Akshita Singh, Prabesh Paudel, Siddhartha Roy · 11. Mai 2026

  We introduce a proxy-analyzer framework for detecting hallucinations in large language models. Instead of looking inside the generating model, our system reads already-generated text through a small locally hosted open-weight model and spots hallucinations using the reader's own internal activations…

• Efficient Verification of Neural Control Barrier Functions with Smooth Nonlinear Activations

  Jun Zhang, Haibo Zhang, Chun Liu, Xiaofan Wang, Liang Xu · 11. Mai 2026

  Formal verification of neural control barrier functions (NCBFs) remains challenging, especially for neural networks with nonlinear activations like \(\tanh\). Existing CROWN-based methods rely on conservative linear relaxations for Jacobian bounds, limiting scalability. We propose LightCROWN, which …