Adversarial Robustness in Machine Learning

Neueste Paper

• Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing

  Mathieu Dario, Florent Chenevier, K\'evin Delmas, Joris Guerin, J\'er\'emie Guiochet · 30. April 2026

  Runtime monitoring is essential to ensure the safety of ML applications in safety-critical domains. However, current research is fragmented, with independent methods emerging from different communities. In this paper, we propose a unified framework categorising runtime monitoring approaches into thr…

• Deep-testing: the case of dependence detection

  Gery Geenens, Pierre Lafaye de Micheaux, Ivan Muyun Zou · 30. April 2026

  Deep learning methods have proved highly effective for classification and image recognition problems. In this paper, we ask whether this success can be transferred to hypothesis testing: if a neural network can distinguish, for example, an image of a handwritten digit from another, can it also disti…

• Value-Guided Iterative Refinement and the DIQ-H Benchmark for Evaluating VLM Robustness

  Hanwen Wan, Zexin Lin, Yixuan Deng, Xiaoqiang Ji · 30. April 2026

  Vision-Language Models (VLMs) are essential for embodied AI and safety-critical applications, such as robotics and autonomous systems. However, existing benchmarks primarily focus on static or curated visual inputs, neglecting the challenges posed by adversarial conditions, value misalignment, and e…

• Test-Time Safety Alignment

  Baturay Saglam, Dionysis Kalogerias · 30. April 2026

  Recent work has shown that a model's input word embeddings can serve as effective control variables for steering its behavior toward outputs that satisfy desired properties. However, this has only been demonstrated for pretrained text-completion models on the relatively simple objective of reducing …

• Adversarial Robustness of NTK Neural Networks

  Yuxuan Hou · 30. April 2026

  Deep learning models are widely deployed in safety-critical domains, but remain vulnerable to adversarial attacks. In this paper, we study the adversarial robustness of NTK neural networks in the context of nonparametric regression. We establish minimax optimal rates for adversarial regression in So…

• Recipes for Calibration Checks in Safety-Critical Applications

  Romeo Valentin · 30. April 2026

  Safety-critical prediction systems, such as autonomous vehicles, weather forecasters, and medical monitors, commonly rely on probabilistic forecasters. These forecasters make predictions about possible future outcomes, and their quality and robustness needs to be validated and certified. Often, only…

• Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics

  Bernd Frauenknecht, Lukas Kesper, Daniel Mayfrank, Henrik Hose, Sebastian Trimpe · 30. April 2026

  Predictive safety filters (PSFs) leverage model predictive control to enforce constraint satisfaction during deep reinforcement learning (RL) exploration, yet their reliance on first-principles models or Gaussian processes limits scalability and broader applicability. Meanwhile, model-based RL (MBRL…

• Poster: ClawdGo: Endogenous Security Awareness Training for Autonomous AI Agents

  Jiaqi Li, Yang Zhao, Bin Sun, Yang Yu, Jian Chang, Lidong Zhai · 29. April 2026

  Autonomous AI agents deployed on platforms such as OpenClaw face prompt injection, memory poisoning, supply-chain attacks, and social engineering, yet existing defences address only the platform perimeter, leaving the agent's own threat judgement entirely untrained. We present ClawdGo, a framework f…

• Evaluation of Prompt Injection Defenses in Large Language Models

  Priyal Deep, Shane Emmons, Amy Fox, Kyle Bacon, Kelley McAllister, Krisztian Flautner · 29. April 2026

  LLM-powered applications routinely embed secrets in system prompts, yet models can be tricked into revealing them. We built an adaptive attacker that evolves its strategies over hundreds of rounds and tested it against nine defense configurations across more than 20,000 attacks. Every defense that r…

• Jailbreaking Frontier Foundation Models Through Intention Deception

  Xinhe Wang, Katia Sycara, Yaqi Xie · 29. April 2026

  Large (vision-)language models exhibit remarkable capability but remain highly susceptible to jailbreaking. Existing safety training approaches aim to have the model learn a refusal boundary between safe and unsafe, based on the user's intent. It has been found that this binary training regime often…

• RouteGuard: Internal-Signal Detection of Skill Poisoning in LLM Agents

  Wenjie Xiao, Xuehai Tang, Biyu Zhou, Songlin Hu, Jizhong Han · 29. April 2026

  Agent skills introduce a new and more severe form of indirect injection for LLM agents: unlike traditional indirect prompt injection, attackers can hide malicious instructions inside a dense, action-oriented skill that already functions as a legitimate instruction source. We study pre-execution skil…

• Mechanistic Steering of LLMs Reveals Layer-wise Feature Vulnerabilities in Adversarial Settings

  Nilanjana Das, Manas Gaur · 29. April 2026

  Large language models (LLMs) can still be jailbroken into producing harmful outputs despite safety alignment. Existing attacks show this vulnerability, but not the internal mechanisms that cause it. This study asks whether jailbreak success is driven by identifiable internal features rather than pro…

• PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attacks

  Tianlong Yu, Yang Yang, Ziyi Zhou, Jiaying Xu, Siwei Li, Tong Guan, Kailong Wang, Ting Bi · 29. April 2026

  The emerging threat of AR-LLM-based Social Engineering (AR-LLM-SE) attacks (e.g. SEAR) poses a significant risk to real-world social interactions. In such an attack, a malicious actor uses Augmented Reality (AR) glasses to capture a target visual and vocal data. A Large Language Model (LLM) then ana…

• Patching LLM Like Software: A Lightweight Method for Improving Safety Policy in Large Language Models

  Huzaifa Arif, Keerthiram Murugesan, Ching-Yun Ko, Pin-Yu Chen, Payel Das, Alex Gittens · 29. April 2026

  We propose patching for large language models (LLMs) like software versions, a lightweight and modular approach for addressing safety vulnerabilities. While vendors release improved LLM versions, major releases are costly, infrequent, and difficult to tailor to customer needs, leaving released model…

• CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks

  Xu Zhang, Hao Li, Zhichao Lu · 29. April 2026

  Multimodal Large Language Models (MLLMs) achieve strong reasoning and perception capabilities but are increasingly vulnerable to jailbreak attacks. While existing work focuses on explicit attacks, where malicious content resides in a single modality, recent studies reveal implicit attacks, in which …

• Right-to-Act: A Pre-Execution Non-Compensatory Decision Protocol for AI Systems

  Gadi Lavi · 29. April 2026

  Current AI systems increasingly operate in contexts where their outputs directly trigger real-world actions. Most existing approaches to AI safety, risk management, and governance focus on post-hoc validation, probabilistic risk estimation, or certification of model behavior. However, these approach…

• Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs

  Byeonggeuk Lim, JungMin Yun, Junehyoung Kwon, Kyeonghyun Kim, YoungBin Kim · 29. April 2026

  Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models…

• Protecting the Trace: A Principled Black-Box Approach Against Distillation Attacks

  Max Hartman, Vidhata Jayaraman, Moulik Choraria, Lav R. Varshney · 29. April 2026

  Frontier models push the boundaries of what is learnable at extreme computational costs, yet distillation via sampling reasoning traces exposes closed-source frontier models to adversarial third parties who can bypass their guardrails and misappropriate their capabilities, raising safety, security, …

• Does Machine Unlearning Preserve Clinical Safety? A Risk Analysis for Medical Image Classification

  Andreza M. C. Falcao, Filipe R. Cordeiro · 29. April 2026

  The application of Deep Learning in medical diagnosis must balance patient safety with compliance with data protection regulations. Machine Unlearning enables the selective removal of training data from deployed models. However, most methods are validated primarily through efficiency and privacy-ori…

• BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks

  Rui Miao, Yixin Liu, Yili Wang, Xu Shen, Yue Tan, Yiwei Dai, Shirui Pan, Xin Wang · 29. April 2026

  The security of LLM-based multi-agent systems (MAS) is critically threatened by propagation vulnerability, where malicious agents can distort collective decision-making through inter-agent message interactions. While existing supervised defense methods demonstrate promising performance, they may be …

• Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty

  Artur Eisele, Bernd Frauenknecht, Friedrich Solowjow, Sebastian Trimpe · 29. April 2026

  Safety remains an open problem in reinforcement learning (RL), especially during training. While safety filters are promising to address safe exploration, they are generally poorly suited for high-dimensional systems with unknown dynamics. We propose Dyna-style Safety Augmented Reinforcement Learnin…

• Defusing the Trigger: Plug-and-Play Defense for Backdoored LLMs via Tail-Risk Intrinsic Geometric Smoothing

  Kaisheng Fan, Weizhe Zhang, Yishu Gao, Tegawend\'e F. Bissyand\'e, Xunzhu Tang · 29. April 2026

  Defending against backdoor attacks in large language models remains a critical practical challenge. Existing defenses mitigate these threats but typically incur high preparation costs and degrade utility via offline purification, or introduce severe latency via complex online interventions. To overc…

• Structural Enforcement of Goal Integrity in AI Agents via Separation-of-Powers Architecture

  Rong Xiang · 29. April 2026

  Recent evidence suggests that frontier AI systems can exhibit agentic misalignment, generating and executing harmful actions derived from internally constructed goals, even without explicit user requests. Existing mitigation methods, such as Reinforcement Learning from Human Feedback (RLHF) and cons…

• Discovering Agentic Safety Specifications from 1-Bit Danger Signals

  V\'ictor Gallego · 29. April 2026

  Can large language model agents discover hidden safety objectives through experience alone? We introduce EPO-Safe (Experiential Prompt Optimization for Safe Agents), a framework where an LLM iteratively generates action plans, receives sparse binary danger warnings, and evolves a natural language be…

• UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks

  Tianlong Yu, Yang Yang, Xiao Luo, Lihong Liu, Fudu Xing, Zui Tao, Kailong Wang, Gaoyang Liu, Ting Bi · 29. April 2026

  Emerging AR-LLM-based Social Engineering attack (e.g., SEAR) is at the edge of posing great threats to real-world social life. In such AR-LLM-SE attack, the attacker can leverage AR (Augmented Reality) glass to capture the image and vocal information of the target, using the LLM to identify the targ…