Adversarial Robustness in Machine Learning

Últimos artículos

• Latent Instruction Representation Alignment: defending against jailbreaks, backdoors and undesired knowledge in LLMs

  Eric Easley, Sebastian Farquhar · 14 de abril de 2026

  We address jailbreaks, backdoors, and unlearning for large language models (LLMs). Unlike prior work, which trains LLMs based on their actions when given malign instructions, our method specifically trains the model to change how it interprets instructions. Our method, Latent Instruction Representat…

• Poisoning with A Pill: Circumventing Detection in Federated Learning

  Hanxi Guo, Hao Wang, Tao Song, Tianhang Zheng, Yang Hua, Haibing Guan, Xiangyu Zhang · 14 de abril de 2026

  Without direct access to the client's data, federated learning (FL) is well-known for its unique strength in data privacy protection among existing distributed machine learning techniques. However, its distributive and iterative nature makes FL inherently vulnerable to various poisoning attacks. To …

• Beyond A Fixed Seal: Adaptive Stealing Watermark in Large Language Models

  Shuhao Zhang, Yuli Chen, Jiale Han, Bo Cheng, Jiabao Ma · 14 de abril de 2026

  Watermarking provides a critical safeguard for large language model (LLM) services by facilitating the detection of LLM-generated text. Correspondingly, stealing watermark algorithms (SWAs) derive watermark information from watermarked texts generated by victim LLMs to craft highly targeted adversar…

• TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection

  Lei Jiang, Chunzhao Xie, Tongxuan Liu, Yuting Zeng, jinrong Guo, Yunheng Shen, Weizhe Huang, Jing Li, Xiaohua Xu · 14 de abril de 2026

  Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of…

• Continuous Adversarial Flow Models

  Shanchuan Lin, Ceyuan Yang, Zhijie Lin, Hao Chen, Haoqi Fan · 14 de abril de 2026

  We propose continuous adversarial flow models, a type of continuous-time flow model trained with an adversarial objective. Unlike flow matching, which uses a fixed mean-squared-error criterion, our approach introduces a learned discriminator to guide training. This change in objective induces a diff…

• FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Models

  Xinyuan An, Tao Luo, Gengyun Peng, Yaobing Wang, Kui Ren, Dongxia Wang · 14 de abril de 2026

  Vision-Language-Action (VLA) models are emerging as a cornerstone for robotics, with flow-matching policies like $\pi_0$ showing great promise in generating smooth, continuous actions. As these models advance, their unique action generation mechanism - the vector field dynamics - presents a critical…

• Safety Guarantees in Zero-Shot Reinforcement Learning for Cascade Dynamical Systems

  Shima Rabiei, Sandipan Mishra, Santiago Paternain · 14 de abril de 2026

  This paper considers the problem of zero-shot safety guarantees for cascade dynamical systems. These are systems where a subset of the states (the inner states) affects the dynamics of the remaining states (the outer states) but not vice-versa. We define safety as remaining on a set deemed safe for …

• Verify Before You Fix: Agentic Execution Grounding for Trustworthy Cross-Language Code Analysis

  Jugal Gajjar · 14 de abril de 2026

  Learned classifiers deployed in agentic pipelines face a fundamental reliability problem: predictions are probabilistic inferences, not verified conclusions, and acting on them without grounding in observable evidence leads to compounding failures across downstream stages. Software vulnerability ana…

• Optimal Stability of KL Divergence under Gaussian Perturbations

  Jialu Pan, Yufeng Zhang, Nan Hu, Keqin Li · 14 de abril de 2026

  We study the problem of characterizing the stability of Kullback-Leibler (KL) divergence under Gaussian perturbations beyond Gaussian families. Existing relaxed triangle inequalities for KL divergence critically rely on the assumption that all involved distributions are Gaussian, which limits their …

• ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection

  Wei Zhao, Zhe Li, Peixin Zhang, Jun Sun · 14 de abril de 2026

  Tool-augmented Large Language Model (LLM) agents have demonstrated impressive capabilities in automating complex, multi-step real-world tasks, yet remain vulnerable to indirect prompt injection. Adversaries exploit this weakness by embedding malicious instructions within tool-returned content, which…

• Robust Adversarial Policy Optimization Under Dynamics Uncertainty

  Mintae Kim, Koushil Sreenath · 14 de abril de 2026

  Reinforcement learning (RL) policies often fail under dynamics that differ from training, a gap not fully addressed by domain randomization or existing adversarial RL methods. Distributionally robust RL provides a formal remedy but still relies on surrogate adversaries to approximate intractable pri…

• Exact Certification of Neural Networks and Partition Aggregation Ensembles against Label Poisoning

  Ajinkya Mohgaonkar, Lukas Gosch, Mahalakshmi Sabanayagam, Debarghya Ghoshdastidar, Stephan G\"unnemann · 14 de abril de 2026

  Label-flipping attacks, which corrupt training labels to induce misclassifications at inference, remain a major threat to supervised learning models. This drives the need for robustness certificates that provide formal guarantees about a model's robustness under adversarially corrupted labels. Exist…

• Deliberative Alignment is Deep, but Uncertainty Remains: Inference time safety improvement in reasoning via attribution of unsafe behavior to base model

  Pankayaraj Pathmanathan, Furong Huang · 14 de abril de 2026

  While the wide adoption of refusal training in large language models (LLMs) has showcased improvements in model safety, recent works have highlighted shortcomings due to the shallow nature of these alignment methods. To this end, the work on Deliberative alignment proposed distilling reasoning capab…

• Hodoscope: Unsupervised Monitoring for AI Misbehaviors

  Ziqian Zhong, Shashwat Saxena, Aditi Raghunathan · 14 de abril de 2026

  Existing approaches to monitoring AI agents rely on supervised evaluation: human-written rules or LLM-based judges that check for known failure modes. However, novel misbehaviors may fall outside predefined categories entirely and LLM-based judges can be unreliable. To address this, we formulate uns…

• Adversarial Robustness of Graph Transformers

  Philipp Foth, Lukas Gosch, Simon Geisler, Leo Schwinn, Stephan G\"unnemann · 14 de abril de 2026

  Existing studies have shown that Message-Passing Graph Neural Networks (MPNNs) are highly susceptible to adversarial attacks. In contrast, despite the increasing importance of Graph Transformers (GTs), their robustness properties are unexplored. We close this gap and design the first adaptive attack…

• Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling

  Keita Broadwater · 14 de abril de 2026

  Traditional benchmarks for large language models (LLMs), such as HELM and AIR-BENCH, primarily assess safety risk through breadth-oriented evaluation across diverse tasks. However, real-world deployment often exposes a different class of risk: operational failures arising from repeated generations o…

• Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion

  Vishal Pramanik, Maisha Maliha, Susmit Jha, Sumit Kumar Jha · 14 de abril de 2026

  Large language models remain vulnerable to jailbreak attacks -- inputs designed to bypass safety mechanisms and elicit harmful responses -- despite advances in alignment and instruction tuning. We propose Head-Masked Nullspace Steering (HMNS), a circuit-level intervention that (i) identifies attenti…

• Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game

  Yuanbo Xie, Yingjie Zhang, Yulin Li, Shouyou Song, Xiaokun Chen, Zhihan Liu, Liya Su, Tingwen Liu · 14 de abril de 2026

  Retrieval-Augmented Generation (RAG) systems augment large language models with external knowledge, yet introduce a critical security vulnerability: RAG Knowledge Base Leakage, wherein adversarial prompts can induce the model to divulge retrieved proprietary content. Recent studies reveal that such …

• What do your logits know? (The answer may surprise you!)

  Masha Fedzechkina, Eleonora Gualdoni, Rita Ramos, Sinead Williamson · 14 de abril de 2026

  Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses the risk of unintentional or malicious information leakage, where model users are able to learn information that the model owner assumed was inaccessible. Using v…

• STARS: Skill-Triggered Audit for Request-Conditioned Invocation Safety in Agent Systems

  Guijia Zhang, Shu Yang, Xilin Gong, Di Wang · 14 de abril de 2026

  Autonomous language-model agents increasingly rely on installable skills and tools to complete user tasks. Static skill auditing can expose capability surface before deployment, but it cannot determine whether a particular invocation is unsafe under the current user request and runtime context. We t…

• Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind

  Hanqi Xiao, Vaidehi Patil, Zaid Khan, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal · 14 de abril de 2026

  As large language models (LLMs) become the engine behind conversational systems, their ability to reason about the intentions and states of their dialogue partners (i.e., form and use a theory-of-mind, or ToM) becomes increasingly critical for safe interaction with potentially adversarial partners. …

• On the Robustness of Watermarking for Autoregressive Image Generation

  Andreas M\"uller, Denis Lukovnikov, Shingo Kodama, Minh Pham, Anubhav Jain, Jonathan Petit, Niv Cohen, Asja Fischer · 14 de abril de 2026

  The proliferation of autoregressive (AR) image generators demands reliable detection and attribution of their outputs to mitigate misinformation, and to filter synthetic images from training data to prevent model collapse. To address this need, watermarking techniques, specifically designed for AR m…

• Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Reward

  Weiyang Guo, Zesheng Shi, Zeen Zhu, Yuan Zhou, Min Zhang, Jing Li · 14 de abril de 2026

  Reinforcement Learning with Verifiable Rewards (RLVR) is an emerging paradigm that significantly boosts a Large Language Model's (LLM's) reasoning abilities on complex logical tasks, such as mathematics and programming. However, we identify, for the first time, a latent vulnerability to backdoor att…

• Mitigating Privacy Risk via Forget Set-Free Unlearning

  Aviraj Newatia, Michael Cooper, Viet Nguyen, Rahul G. Krishnan · 14 de abril de 2026

  Training machine learning models requires the storage of large datasets, which often contain sensitive or private data. Storing data is associated with a number of potential risks which increase over time, such as database breaches and malicious adversaries. Machine unlearning is the study of method…

• The Deployment Gap in AI Media Detection: Platform-Aware and Visually Constrained Adversarial Evaluation

  Aishwarya Budhkar, Trishita Dhara, Siddhesh Sheth · 14 de abril de 2026

  Recent AI media detectors report near-perfect performance under clean laboratory evaluation, yet their robustness under realistic deployment conditions remains underexplored. In practice, AI-generated images are resized, compressed, re-encoded, and visually modified before being shared on online pla…