Adversarial Robustness in Machine Learning

Neueste Paper

• PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models

  Simone Gallivanone, Hossein Khodadadi, Mauro Dore, Mauro Medda, Nicola Franco · 24. Juni 2026

  We introduce a large-scale, open-source dataset of pre-generated adversarial attacks for vision-language models (VLMs). The dataset is designed to be diverse, representative, and practical, extending existing benchmarks by covering 10 high-level categories and 55 subcategories of harmful intents. Ou…

• Are Safety Guarantees in Neural Networks Safe? How to Compute Trustworthy Robustness Certifications

  Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris, Jo\~ao Marques-Silva · 24. Juni 2026

  A primary challenge in AI safety is the existence of adversarial examples -- slightly distorted inputs that cause a neural network (NN) to misclassify. To mitigate this problem, recent research focuses on the computation of robustness certifications, which, for a given input, determine the largest d…

• RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems

  Yarin Yerushalmi Levi, Roy Betser, Amit Giloni, Lidor Erez, Itay Gershon, Oren Rachmil, Sindhu Padakandla, Roman Vainshtein · 24. Juni 2026

  Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities. Existing security evaluations are often tied to specific implementations or domains, limiting unified comp…

• Reinforcement Learning Towards Broadly and Persistently Beneficial Models

  Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab, Ali Malik, Mikhail Trofimov, Foivos Tsimpourlas, Johannes Heidecke, Karan Singhal · 24. Juni 2026

  As AI systems are deployed across increasingly diverse and high-stakes settings, model alignment must generalize beyond the tasks and domains seen during training. This is especially important for reinforcement learning (RL), which can introduce unexpected misalignment through reward hacking, decept…

• Grad Detect: Gradient-Based Hallucination Detection in LLMs

  Anand Kamat, Daniel Blake, Brent M. Werness · 24. Juni 2026

  Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, yet they remain prone to generating hallucinations. Detecting these hallucinations is critical for deploying LLMs reliably in high-stakes applications. We present Grad Detect, a gradient-based approach for p…

• When Top-1 Fails: Calibrating LoRA Monitors for Masked Diffusion LMs

  Lucky Verma, Pratik Yadav · 24. Juni 2026

  Discrete diffusion language model (DLM) fine-tuning inherits inexpensive diagnostics from denoising-time confidence monitors, but their PEFT-training meaning is untested. We test top-1 argmax concentration as a collapse warning. Across 816 LoRA/PEFT configurations from three DLM families, the warnin…

• BARD-MARL: Byzantine-Agent Detection for Learned Communication in Multi-Agent Reinforcement Learning

  Almond Kiruthu Murimi · 23. Juni 2026

  Learned communication improves coordination in cooperative multi-agent reinforcement learning, but it also creates a trust problem: a trained policy may route information through agents that have become faulty or adversarial. This paper studies Byzantine-agent detection for learned-communication MAR…

• NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms

  Hanwool Lee, Dasol Choi, Bokyeong Kim, Seung Geun Kim, Haon Park · 23. Juni 2026

  Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized. We present NRT-Bench, a benchmark for multi-turn red-teaming of LLM agents acting as op…

• Adversarial observations in probabilistic State-Space Models for robust Reinforcement Learning

  M. Santos-Pascual, D. R\'ios Insua · 23. Juni 2026

  Decision-making under partial or adversarial observability requires accurate inference of the environment's latent state and its associated uncertainty. This work analyses adversarial attacks on linear probabilistic state-space models, commonly integrated within reinforcement learning architectures,…

• Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents

  Wangxuan Fan, Xiaoyu Nie, Zhongxiang Dai · 23. Juni 2026

  The increasing deployment of large language model (LLM) agents in collaborative workflows demands robust multi-user, multi-principal interaction mechanisms capable of enforcing access permissions, resolving authoritative conflicts, and preventing unauthorized data disclosure. However, a fundamental …

• AdaReP:Adaptive Re-Planning under Model Mismatch for Neural World-Model Predictive Control

  Yutian Cheng, Xiaojian Ma, Xianhao Wang, Min Yang, Rongpeng Su, Hangxin Liu, Xi Chen, Shuai Li, Qing Li · 23. Juni 2026

  Neural world models coupled with model predictive control (MPC) replan at every environment step to bound accumulated prediction error, but this incurs substantial computational overhead. Reusing a cached plan reduces this overhead, yet its effectiveness depends on how prediction mismatch propagates…

• Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness

  Evan Duan · 23. Juni 2026

  Activation monitors -- lightweight probes trained on a language model's internal representations -- are an increasingly common layer in deployment safety stacks. Deployed models however are rarely static: they are quantized, fine-tuned, adapted with LoRA, or served with merged adapters while the mon…

• CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes

  Zhijian Zhou, Zesheng Ye, Zhaorun Chen, Bo Li, Feng Liu · 23. Juni 2026

  Can we trust evaluation scores to capture an LLM's true real-world performance? Certifiable evaluation answers this question by providing guarantee for LLM evaluation. In particular, existing methods sequentially curate evaluation samples and keep updating confidence intervals (CIs) that cover the t…

• When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents

  Yanhang Li, Zhichao Fan, Zexin Zhuang · 23. Juni 2026

  Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt injection (IPI) in multimodal computer-use agents before the agent emits a corrupted action. We argue, on a single-backbon…

• Delay-Adaptive Speculation Control for Low-Latency Edge-Cloud LLM Inference

  Kangkang Sun, Jianhua Li, Xiuzhen Chen, Junyi He, Minyi Guo · 23. Juni 2026

  Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a larger target model to verify them in parallel. In distributed edge-cloud inference, however, draft length must be controlled online: longer drafts amortize communication …

• Reliability-Guided Adaptive Ensembling for Robust Test-Time Adaptation

  Adam Koziak, Yuhong Guo · 23. Juni 2026

  Test-time adaptation (TTA) can mitigate domain shift without source data, but it is highly brittle under adversarially contaminated test streams, where corrupted inputs also destabilize online updates. We study robust test-time adaptation (RTTA) in the adversarial-stream setting, which remains compa…

• Efficient Safety Benchmarking via Item Response Theory

  Fabio Spagliardi, M\'irian Silva, Ayan Datta, Aiden Zhou, Vamshi Bonagiri, Diogo Cruz · 23. Juni 2026

  Safety benchmarks for language models are typically evaluated using static paradigms that treat all items as equally informative for all models, an assumption that is particularly problematic for adversarial, highly heterogeneous safety items. Applied in full to modern benchmark suites, the current …

• SkillHarness: Harnessing Safe Skills for Computer-Use Agents

  Yurun Chen, Biao Yi, Keting Yin, Shengyu Zhang · 23. Juni 2026

  Computer-Use Agents (CUAs) are increasingly deployed in dynamic interactive environments, creating a growing need for continual skill learning during interaction. Recent approaches address this challenge by learning reusable skills from successful trajectories. However, these skill learning methods …

• AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agent

  Weidi Luo, Qiming Zhang, Yihao Quan, Mingyu Jin, Jie Cai, Chaowei Xiao, Jingcheng Niu, Zhen Xiang · 23. Juni 2026

  Coding agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities, but they also introduce significant safety and misuse risks during multi-turn interactions with external environments. Existing safety mechanisms mainly rely on external guardrails, which have a limit…

• Skin-Deep: A Geometric Diagnostic for Alignment Fragility in Large Language Model Representations

  Dongyub Jude Lee, Jungseob Lee, Seungyoon Lee, Seongtae Hong, Suhyune Son, Sugyeong Eo, Jaehyung Seo, Heuiseok Lim · 23. Juni 2026

  Alignment tuning is meant to make harmful-request refusal robust, yet this safety behavior can be erased by a small set of benign fine-tuning examples. This is a deployment risk for open-weight models because a checkpoint can pass refusal tests at release time and later lose refusal under low-cost d…

• MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents

  Xuelong Dai, Jianyu Ma, Boyang Ma, Biwei Yan, Yijun Yang, Yue Zhang · 23. Juni 2026

  Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing adversarial evaluations targeting these agents frequently re…

• LambdaMark: Semantic Audio Watermarking for Robustness and Radioactivity

  Kexin Li, Xiao Hu, Ilya Grishchenko, David Lie · 23. Juni 2026

  Recent advances in generative audio have made voice cloning increasingly effortless, enabling voice fraud, impersonation, and other forms of unauthorized use. A common attack finetunes a speech generation model on recordings of a target speaker, allowing the model to synthesize speech in that speake…

• Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models

  Linghan Chen, Kaiyan Ji, Minyu Guo · 23. Juni 2026

  Many recent vision-language-action (VLA) policies adopt an imagine-then-act design. A world-action model (WAM) first imagines a short future as a latent trajectory z~, on which the action is then conditioned. We identify this trusted imagination, rather than the reactive policy, as the exposed attac…

• MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning

  Weile Guo, Shenghong He, Danying Mo, Chengdong Xu, Xuexun Liu, Chao Yu · 23. Juni 2026

  Motion instruction generation in cross-video comparison aims to produce corrective feedback that describes the differences between a query and a reference motion. However, existing models often generate instructions that exhibit motion hallucinations, failing to reflect actual kinematic differences …

• Exposing the Illusion of Erasure in Knowledge Editing for LLMs

  Advik Raj Basani, Anshuman Chhabra · 23. Juni 2026

  Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood. In this work, we examine KE from an adversarial elicitation perspective, revealing that edited knowledge is often no…