Adversarial Robustness in Machine Learning

Últimos artículos

• Re-Mask and Redirect: Exploiting Denoising Irreversibility in Diffusion Language Models

  Arth Singh · 13 de abril de 2026

  Diffusion-based language models (dLLMs) generate text by iteratively denoising masked token sequences. We show that their safety alignment rests on a single fragile assumption: that the denoising schedule is monotonic and committed tokens are never re-evaluated. Safety-aligned dLLMs commit refusal t…

• GRM: Utility-Aware Jailbreak Attacks on Audio LLMs via Gradient-Ratio Masking

  Yunqiang Wang, Hengyuan Na, Di Wu, Miao Hu, Guocong Quan · 13 de abril de 2026

  Audio large language models (ALLMs) enable rich speech-text interaction, but they also introduce jailbreak vulnerabilities in the audio modality. Existing audio jailbreak methods mainly optimize jailbreak success while overlooking utility preservation, as reflected in transcription quality and quest…

• Reducing Class Bias In Data-Balanced Datasets Through Hardness-Based Resampling

  Pawel Pukowski, Venet Osmani · 13 de abril de 2026

  Class-bias, that is class-wise performance disparities, is typically attributed to data imbalance and addressed through frequency-based resampling. However, we demonstrate that substantial bias persists even in perfectly balanced datasets, proving that class frequency alone cannot explain unequal mo…

• Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer

  Yannis Montreuil, Axel Carlier, Lai Xing Ng, Wei Tsang Ooi · 13 de abril de 2026

  Learning-to-Defer routes each input to the expert that minimizes expected cost, but it assumes that the information available to every expert is fixed at decision time. Many modern systems violate this assumption: after selecting an expert, one may also choose what additional information that expert…

• Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance

  Roi Paul · 13 de abril de 2026

  We study whether low-rank spectral summaries of LoRA weight deltas can identify which fine-tuning objective was applied to a language model, and whether that geometric signal predicts downstream behavioral harm. In a pre-registered experiment on \texttt{Llama-3.2-3B-Instruct}, we manufacture 38 LoRA…

• Mosaic: Multimodal Jailbreak against Closed-Source VLMs via Multi-View Ensemble Optimization

  Yuqin Lan, Gen Li, Yuanze Hu, Weihao Shen, Zhaoxin Fan, Faguo Wu, Xiao Zhang, Laurence T. Yang, Zhiming Zheng · 13 de abril de 2026

  Vision-Language Models (VLMs) are powerful but remain vulnerable to multimodal jailbreak attacks. Existing attacks mainly rely on either explicit visual prompt attacks or gradient-based adversarial optimization. While the former is easier to detect, the latter produces subtle perturbations that are …

• Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection

  Zedian Shao, Hongbin Liu, Yuepeng Hu, Neil Zhenqiang Gong · 13 de abril de 2026

  Multi-modal large language models (MLLMs) have emerged as powerful tools for analyzing Internet-scale image data, offering significant benefits but also raising critical safety and societal concerns. In particular, open-weight MLLMs may be misused to extract sensitive information from personal image…

• Thermally Activated Dual-Modal Adversarial Clothing against AI Surveillance Systems

  Jiahuan Long, Tingsong Jiang, Hanqing Liu, Chao Ma, Weien Zhou, Yang Yang, Wen Yao · 13 de abril de 2026

  Adversarial patches have emerged as a popular privacy-preserving approach for resisting AI-driven surveillance systems. However, their conspicuous appearance makes them difficult to deploy in real-world scenarios. In this paper, we propose a thermally activated adversarial wearable designed to ensur…

• Enhancing the Safety of Medical Vision-Language Models by Synthetic Demonstrations

  Zhiyu Xue, Reza Abbasi-Asl, Ramtin Pedarsani · 13 de abril de 2026

  Generative medical vision-language models~(Med-VLMs) are primarily designed to generate complex textual information~(e.g., diagnostic reports) from multimodal inputs including vision modality~(e.g., medical images) and language modality~(e.g., clinical queries). However, their security vulnerabiliti…

• Are Independently Estimated View Uncertainties Comparable? Unified Routing for Trusted Multi-View Classification

  Yilin Zhang, Cai Xu, Haishun Chen, Ziyu Guan, Wei Zhao · 13 de abril de 2026

  Trusted multi-view classification typically relies on a view-wise evidential fusion process: each view independently produces class evidence and uncertainty, and the final prediction is obtained by aggregating these independent opinions. While this design is modular and uncertainty-aware, it implici…

• XFED: Non-Collusive Model Poisoning Attack Against Byzantine-Robust Federated Classifiers

  Israt Jahan Mouri, Muhammad Ridowan, Muhammad Abdullah Adnan · 13 de abril de 2026

  Model poisoning attacks pose a significant security threat to Federated Learning (FL). Most existing model poisoning attacks rely on collusion, requiring adversarial clients to coordinate by exchanging local benign models and synchronizing the generation of their poisoned updates. However, sustainin…

• CLIP-Inspector: Model-Level Backdoor Detection for Prompt-Tuned CLIP via OOD Trigger Inversion

  Akshit Jindal, Saket Anand, Chetan Arora, Vikram Goyal · 13 de abril de 2026

  Organisations with limited data and computational resources increasingly outsource model training to Machine Learning as a Service (MLaaS) providers, who adapt vision-language models (VLMs) such as CLIP to downstream tasks via prompt tuning rather than training from scratch. This semi-honest setting…

• From Dispersion to Attraction: Spectral Dynamics of Hallucination Across Whisper Model Scales

  Ivan Viakhirev, Kirill Borodin, Grach Mkrtchian · 13 de abril de 2026

  Hallucinations in large ASR models present a critical safety risk. In this work, we propose the \textit{Spectral Sensitivity Theorem}, which predicts a phase transition in deep networks from a dispersive regime (signal decay) to an attractor regime (rank-1 collapse) governed by layer-wise gain and a…

• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation

  Yushi Feng, Junye Du, Qifan Wang, Zizhan Ma, Qian Niu, Yutaka Matsuo, Long Feng, Lequan Yu · 13 de abril de 2026

  Graphical user interface (GUI) agents powered by vision language models (VLMs) are rapidly moving from passive assistance to autonomous operation. However, this unrestricted action space exposes users to severe and irreversible financial, privacy or social harm. Existing safeguards rely on prompt en…

• PilotBench: A Benchmark for General Aviation Agents with Safety Constraints

  Yalun Wu, Haotian Liu, Zhoujun Li, Boyang Wang · 13 de abril de 2026

  As Large Language Models (LLMs) advance toward embodied AI agents operating in physical environments, a fundamental question emerges: can models trained on text corpora reliably reason about complex physics while adhering to safety constraints? We address this through PilotBench, a benchmark evaluat…

• OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chains

  Jun He, Deying Yu · 13 de abril de 2026

  The rise of autonomous AI agents exposes a fundamental flaw in API-centric architectures: probabilistic systems directly execute state mutations without sufficient context, coordination, or safety guarantees. We introduce OpenKedge, a protocol that redefines mutation as a governed process rather tha…

• Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism

  Hadas Orgad, Boyi Wei, Kaden Zheng, Martin Wattenberg, Peter Henderson, Seraphina Goldfarb-Tarrant, Yonatan Belinkov · 13 de abril de 2026

  Large language models (LLMs) undergo alignment training to avoid harmful behaviors, yet the resulting safeguards remain brittle: jailbreaks routinely bypass them, and fine-tuning on narrow domains can induce ``emergent misalignment'' that generalizes broadly. Whether this brittleness reflects a fund…

• VOLTA: The Surprising Ineffectiveness of Auxiliary Losses for Calibrated Deep Learning

  Rahul D Ray, Utkarsh Srivastava · 13 de abril de 2026

  Uncertainty quantification (UQ) is essential for deploying deep learning models in safety critical applications, yet no consensus exists on which UQ method performs best across different data modalities and distribution shifts. This paper presents a comprehensive benchmark of ten widely used UQ base…

• Towards provable probabilistic safety for scalable embodied AI systems

  Linxuan He, Lingxiang Fan, Qing-Shan Jia, Ang Li, Hongyan Sang, Ling Wang, Guanghui Wen, Jiwen Lu, Tao Zhang, Jie Zhou, Yi Zhang, Yisen Wang, Peng Wei, Zhongyuan Wang, Henry X. Liu, Shuo Feng · 10 de abril de 2026

  Embodied AI systems, comprising AI models and physical plants, are increasingly prevalent across various applications. Due to the rarity of system failures, ensuring their safety in complex operating environments remains a major challenge, which severely hinders their large-scale deployment in safet…

• Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitization

  Igor Maljkovic, Maria Rosaria Briglia, Iacopo Masi, Antonio Emanuele Cin\`a, Fabio Roli · 10 de abril de 2026

  Vision-Language Models (VLMs) have become essential for tasks such as image synthesis, captioning, and retrieval by aligning textual and visual information in a shared embedding space. Yet, this flexibility also makes them vulnerable to malicious prompts designed to produce unsafe content, raising c…

• SALLIE: Safeguarding Against Latent Language & Image Exploits

  Guy Azov, Ofer Rivlin, Guy Shtar · 10 de abril de 2026

  Large Language Models (LLMs) and Vision-Language Models (VLMs) remain highly vulnerable to textual and visual jailbreaks, as well as prompt injections (arXiv:2307.15043, Greshake et al., 2023, arXiv:2306.13213). Existing defenses often degrade performance through complex input transformations or tre…

• ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis

  Yu Li, Haoyu Luo, Yuejin Xie, Yuqian Fu, Zhonghao Yang, Shuai Shao, Qihan Ren, Wanying Qu, Yanwei Fu, Yujiu Yang, Jing Shao, Xia Hu, Dongrui Liu · 10 de abril de 2026

  Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or final responses. Existing trajectory-level benchmarks remain limited by insufficient interaction diversity, coarse obse…

• Data Leakage in Automotive Perception: Practitioners' Insights

  Md Abu Ahammed Babu, Sushant Kumar Pandey, Darko Durisic, Andras Balint, Miroslaw Staron · 10 de abril de 2026

  Data leakage is the inadvertent transfer of information between training and evaluation datasets that poses a subtle, yet critical, risk to the reliability of machine learning (ML) models in safety-critical systems such as automotive perception. While leakage is widely recognized in research, little…

• ClawLess: A Security Model of AI Agents

  Hongyi Lu, Nian Liu, Shuai Wang, Fengwei Zhang · 10 de abril de 2026

  Autonomous AI agents powered by Large Language Models can reason, plan, and execute complex tasks, but their ability to autonomously retrieve information and run code introduces significant security risks. Existing approaches attempt to regulate agent behavior through training or prompting, which do…

• SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills

  Yinghan Hou, Zongyou Yang · 10 de abril de 2026

  OpenClaw's ClawHub marketplace hosts over 13,000 community-contributed agent skills, and between 13% and 26% of them contain security vulnerabilities according to recent audits. Regex scanners miss obfuscated payloads; formal static analyzers cannot read the natural language instructions in SKILL.md…