Adversarial Robustness in Machine Learning

Últimos artículos

• BadLLM-TG: A Backdoor Defender powered by LLM Trigger Generator

  Ruyi Zhang, Heng Gao, Songlei Jian, Yusong Tan, Haifang Zhou · 18 de marzo de 2026

  Backdoor attacks compromise model reliability by using triggers to manipulate outputs. Trigger inversion can accurately locate these triggers via a generator and is therefore critical for backdoor defense. However, the discrete nature of text prevents existing noise-based trigger generator from bein…

• Efficient LLM Safety Evaluation through Multi-Agent Debate

  Dachuan Lin, Guobin Shen, Zihao Yang, Tianrong Liu, Dongcheng Zhao, Yi Zeng · 18 de marzo de 2026

  Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-Judge frameworks, but the high cost of frontier models limits scalability. We propose a cost-efficient multi-agent judging framework that employs Small Language Models (SLMs) through structured debates among critic, de…

• Kestrel: Grounding Self-Refinement for LVLM Hallucination Mitigation

  Jiawei Mao, Hardy Chen, Haoqin Tu, Yuhan Wang, Letian Zhang, Zeyu Zheng, Huaxiu Yao, Zirui Wang, Cihang Xie, Yuyin Zhou · 18 de marzo de 2026

  Large vision-language models (LVLMs) have become increasingly strong but remain prone to hallucinations in multimodal tasks, which significantly narrows their deployment. As training these LVLMs to avoid hallucinations becomes prohibitively expensive for larger models, training-free methods offer a …

• How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition

  Mateusz Dziemian, Maxwell Lin, Xiaohan Fu, Micha Nowak, Nick Winter, Eliot Jones, Andy Zou, Lama Ahmad, Kamalika Chaudhuri, Sahana Chennabasappa, Xander Davies, Lauren Deason, Benjamin L. Edelman, Tanner Emek, Ivan Evtimov, Jim Gust, Maia Hamin, Kat He, Klaudia Krawiecka, Riccardo Patana, Neil Perry, Troy Peterson, Xiangyu Qi, Javier Rando, Zifan Wang, Zihan Wang, Spencer Whitman, Eric Winsor, Arman Zharmagambetov, Matt Fredrikson, Zico Kolter · 18 de marzo de 2026

  LLM based agents are increasingly deployed in high stakes settings where they process external data sources such as emails, documents, and code repositories. This creates exposure to indirect prompt injection attacks, where adversarial instructions embedded in external content manipulate agent behav…

• DynaTrust: Defending Multi-Agent Systems Against Sleeper Agents via Dynamic Trust Graphs

  Yu Li, Qiang Hu, Yao Zhang, Lili Quan, Jiongchi Yu, Junjie Wang · 18 de marzo de 2026

  Large Language Model-based Multi-Agent Systems (MAS) have demonstrated remarkable collaborative reasoning capabilities but introduce new attack surfaces, such as the sleeper agent, which behave benignly during routine operation and gradually accumulate trust, only revealing malicious behaviors when …

• Attribution-Guided Model Rectification of Unreliable Neural Network Behaviors

  Peiyu Yang, Naveed Akhtar, Jiantong Jiang, Ajmal Mian · 18 de marzo de 2026

  The performance of neural network models deteriorates due to their unreliable behavior on non-robust features of corrupted samples. Owing to their opaque nature, rectifying models to address this problem often necessitates arduous data cleaning and model retraining, resulting in huge computational a…

• IdentityGuard: Context-Aware Restriction and Provenance for Personalized Synthesis

  Lingyun Zhang, Yu Xie, Ping Chen · 18 de marzo de 2026

  The nature of personalized text-to-image models poses a unique safety challenge that generic context-blind methods are ill-equipped to handle. Such global filters create a dilemma: to prevent misuse, they are forced to damage the model's broader utility by erasing concepts entirely, causing unaccept…

• REFORGE: Multi-modal Attacks Reveal Vulnerable Concept Unlearning in Image Generation Models

  Yong Zou, Haoran Li, Fanxiao Li, Shenyang Wei, Yunyun Dong, Li Tang, Wei Zhou, Renyang Liu · 18 de marzo de 2026

  Recent progress in image generation models (IGMs) enables high-fidelity content creation but also amplifies risks, including the reproduction of copyrighted content and the generation of offensive content. Image Generation Model Unlearning (IGMU) mitigates these risks by removing harmful concepts wi…

• MOSAIC: Composable Safety Alignment with Modular Control Tokens

  Jingyu Peng, Hongyu Chen, Jiancheng Dong, Maolin Wang, Wenxi Li, Yuchen Li, Kai Zhang, Xiangyu Zhao · 18 de marzo de 2026

  Safety alignment in large language models (LLMs) is commonly implemented as a single static policy embedded in model parameters. However, real-world deployments often require context-dependent safety rules that vary across users, regions, and applications. Existing approaches struggle to provide suc…

• EcoAlign: An Economically Rational Framework for Efficient LVLM Alignment

  Ruoxi Cheng, Haoxuan Ma, Teng Ma, Hongyi Zhang · 17 de marzo de 2026

  Large Vision-Language Models (LVLMs) exhibit powerful reasoning capabilities but suffer sophisticated jailbreak vulnerabilities. Fundamentally, aligning LVLMs is not just a safety challenge but a problem of economic efficiency. Current alignment methods struggle with the trade-off between safety, ut…

• The Active Discoverer Framework: Towards Autonomous Physics Reasoning through Neuro-Symbolic LaTeX Synthesis

  Hyunjun Jeon · 17 de marzo de 2026

  Modern artificial intelligence excels at statistical interpolation within seen manifolds but fundamentally fails at the exact reasoning required for theoretical physics and mathematics. We identify the "Float Wall" -- a catastrophic collapse of neural extrapolation at scales beyond $10^{16}$ -- caus…

• Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projection

  Yewon Han, Yumin Seol, EunGyung Kong, Minsoo Jo, Taesup Kim · 17 de marzo de 2026

  Existing jailbreak defence frameworks for Large Vision-Language Models often suffer from a safety utility tradeoff, where strengthening safety inadvertently degrades performance on general visual-grounded reasoning tasks. In this work, we investigate whether safety and utility are inherently antagon…

• SimCert: Probabilistic Certification for Behavioral Similarity in Deep Neural Network Compression

  Jingyang Li, Fu Song, Guoqiang Li · 17 de marzo de 2026

  Deploying Deep Neural Networks (DNNs) on resource-constrained embedded systems requires aggressive model compression techniques like quantization and pruning. However, ensuring that the compressed model preserves the behavioral fidelity of the original design is a critical challenge in the safety-cr…

• RESQ: A Unified Framework for REliability- and Security Enhancement of Quantized Deep Neural Networks

  Ali Soltan Mohammadi, Samira Nazari, Ali Azarpeyvand, Mahdi Taheri, Milos Krstic, Michael Huebner, Christian Herglotz, Tara Ghasempouri · 17 de marzo de 2026

  This work proposes a unified three-stage framework that produces a quantized DNN with balanced fault and attack robustness. The first stage improves attack resilience via fine-tuning that desensitizes feature representations to small input perturbations. The second stage reinforces fault resilience …

• SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration

  Yu Pan, Wenlong Yu, Tiejun Wu, Xiaohu Ye, Qiannan Si, Guangquan Xu, Bin Wu · 17 de marzo de 2026

  Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks. However, they remain highly susceptible to jailbreak attacks that undermine their safety alignment. Existing defense mechanisms typically rely on post hoc filtering applied only to the final output, le…

• Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinking

  Zichong Wang, Yang Zhou, David Lo, Wenbin Zhang · 17 de marzo de 2026

  The increasing use of Machine Learning (ML) software can lead to unfair and unethical decisions, thus fairness bugs in software are becoming a growing concern. Addressing these fairness bugs often involves sacrificing ML performance, such as accuracy. To address this issue, we present a novel counte…

• CtrlAttack: A Unified Attack on World-Model Control in Diffusion Models

  Shuhan Xu, Siyuan Liang, Hongling Zheng, Yong Luo, Han Hu, Lefei Zhang, Dacheng Tao · 17 de marzo de 2026

  Diffusion-based image-to-video (I2V) models increasingly exhibit world-model-like properties by implicitly capturing temporal dynamics. However, existing studies have mainly focused on visual quality and controllability, and the robustness of the state transition learned by the model remains underst…

• Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective

  Osvaldo Simeone, Yaniv Romano · 17 de marzo de 2026

  In context-specific applications such as robotics, telecommunications, and healthcare, artificial intelligence systems often face the challenge of limited training data. This scarcity introduces epistemic uncertainty, i.e., reducible uncertainty stemming from incomplete knowledge of the underlying d…

• Accelerating Suffix Jailbreak attacks with Prefix-Shared KV-cache

  Xinhai Wang, Shaopeng Fu, Shu Yang, Liangyu Wang, Tianhang Zheng, Di Wang · 17 de marzo de 2026

  Suffix jailbreak attacks serve as a systematic method for red-teaming Large Language Models (LLMs) but suffer from prohibitive computational costs, as a large number of candidate suffixes need to be evaluated before identifying a jailbreak suffix. This paper presents Prefix-Shared KV Cache (PSKV), a…

• Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs

  Zijian Ling, Pingyi Hu, Xiuyong Gao, Xiaojing Ma, Man Zhou, Jun Feng, Songfeng Lu, Dongmei Zhang, Bin Benjamin Zhu · 17 de marzo de 2026

  Speech-driven large language models (LLMs) are increasingly accessed through speech interfaces, introducing new security risks via open acoustic channels. We present Sirens' Whisper (SWhisper), the first practical framework for covert prompt-based attacks against speech-driven LLMs under realistic b…

• ILION: Deterministic Pre-Execution Safety Gates for Agentic AI Systems

  Florin Adrian Chitan · 17 de marzo de 2026

  The proliferation of autonomous AI agents capable of executing real-world actions - filesystem operations, API calls, database modifications, financial transactions - introduces a class of safety risk not addressed by existing content-moderation infrastructure. Current text-safety systems evaluate l…

• Agent Privilege Separation in OpenClaw: A Structural Defense Against Prompt Injection

  Darren Cheng, Wen-Kwang Tsao · 17 de marzo de 2026

  Prompt injection remains one of the most practical attack vectors against LLM-integrated applications. We replicate the Microsoft LLMail-Inject benchmark (Greshake et al., 2024) against current generation models running inside OpenClaw, an open source multitool agent platform. Our proposed defense c…

• AI Evasion and Impersonation Attacks on Facial Re-Identification with Activation Map Explanations

  Noe Claudel, Weisi Guo, Yang Xing · 17 de marzo de 2026

  Facial identification systems are increasingly deployed in surveillance and yet their vulnerability to adversarial evasion and impersonation attacks pose a critical risk. This paper introduces a novel framework for generating adversarial patches capable of both evasion and impersonation attacks agai…

• Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph

  Zhenheng Tang, Xiang Liu, Qian Wang, Eunsol Choi, Bo Li, Xiaowen Chu · 17 de marzo de 2026

  As Large Language Models (LLMs) become more powerful and autonomous, they increasingly face conflicts and dilemmas in many scenarios. We first summarize and taxonomize these diverse conflicts. Then, we model the LLM's preferences to make different choices as a priority graph, where instructions and …

• AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation

  Changyi Li, Pengfei Lu, Xudong Pan, Fazl Barez, Min Yang · 17 de marzo de 2026

  As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators are scalable but suffer from logic hallucination. We present AutoControl Arena, an automated framework for frontier AI ris…