Adversarial Robustness in Machine Learning

Últimos artículos

• Automated Membership Inference Attacks: Discovering MIA Signal Computations using LLM Agents

  Toan Tran, Olivera Kotevska, Li Xiong · 23 de marzo de 2026

  Membership inference attacks (MIAs), which enable adversaries to determine whether specific data points were part of a model's training dataset, have emerged as an important framework to understand, assess, and quantify the potential information leakage associated with machine learning systems. Desi…

• When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models

  Zafir Shamsi, Nikhil Chekuru, Zachary Guzman, Shivank Garg · 23 de marzo de 2026

  Large Language Models (LLMs) are increasingly integrated into high-stakes applications, making robust safety guarantees a central practical and commercial concern. Existing safety evaluations predominantly rely on fixed collections of harmful prompts, implicitly assuming non-adaptive adversaries and…

• Evolving Jailbreaks: Automated Multi-Objective Long-Tail Attacks on Large Language Models

  Wenjing Hong, Zhonghua Rong, Li Wang, Feng Chang, Jian Zhu, Ke Tang, Zexuan Zhu, Yew-Soon Ong · 23 de marzo de 2026

  Large Language Models (LLMs) have been widely deployed, especially through free Web-based applications that expose them to diverse user-generated inputs, including those from long-tail distributions such as low-resource languages and encrypted private data. This open-ended exposure increases the ris…

• The Autonomy Tax: Defense Training Breaks LLM Agents

  Shawn Li, Yue Zhao · 23 de marzo de 2026

  Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through …

• KoALA: KL-L0 Adversarial Detector via Label Agreement

  Siqi Li, Yasser Shoukry · 23 de marzo de 2026

  Deep neural networks are highly susceptible to adversarial attacks, which pose significant risks to security- and safety-critical applications. We present KoALA (KL-L0 Adversarial detection via Label Agreement), a novel, semantics-free adversarial detector that requires no architectural changes or a…

• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination

  Dong-Xiao Zhang, Hu Lou, Jun-Jie Zhang, Jun Zhu, Deyu Meng · 23 de marzo de 2026

  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems, each addressed with modality-specific patches. This study first reveals that they share a common geometric origin: the input and its loss gradient are conjugate observables …

• A Concept is More Than a Word: Diversified Unlearning in Text-to-Image Diffusion Models

  Duc Hao Pham, Van Duy Truong, Duy Khanh Dinh, Tien Cuong Nguyen, Dien Hy Ngo, Tuan Anh Bui · 20 de marzo de 2026

  Concept unlearning has emerged as a promising direction for reducing the risks of harmful content generation in text-to-image diffusion models by selectively erasing undesirable concepts from a model's parameters. Existing approaches typically rely on keywords to identify the target concept to be un…

• Box Maze: A Process-Control Architecture for Reliable LLM Reasoning

  Zou Qiang · 20 de marzo de 2026

  Large language models (LLMs) demonstrate strong generative capabilities but remain vulnerable to hallucination and unreliable reasoning under adversarial prompting. Existing safety approaches -- such as reinforcement learning from human feedback (RLHF) and output filtering -- primarily operate at th…

• CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring

  Jin Mo Yang, Hyung-Sin Kim, Saewoong Bahk · 20 de marzo de 2026

  Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitati…

• Revisiting Label Inference Attacks in Vertical Federated Learning: Why They Are Vulnerable and How to Defend

  Yige Liu, Dexuan Xu, Zimai Guo, Yongzhi Cao, Hanpin Wang · 20 de marzo de 2026

  Vertical federated learning (VFL) allows an active party with a top model, and multiple passive parties with bottom models to collaborate. In this scenario, passive parties possessing only features may attempt to infer active party's private labels, making label inference attacks (LIAs) a significan…

• Beyond Passive Aggregation: Active Auditing and Topology-Aware Defense in Decentralized Federated Learning

  Sheng Pan, Niansheng Tang · 20 de marzo de 2026

  Decentralized Federated Learning (DFL) remains highly vulnerable to adaptive backdoor attacks designed to bypass traditional passive defense metrics. To address this limitation, we shift the defensive paradigm toward a novel active, interventional auditing framework. First, we establish a dynamical …

• A Computationally Efficient Learning of Artificial Intelligence System Reliability Considering Error Propagation

  Fenglian Pan, Yinwei Zhang, Yili Hong, Larry Head, Jian Liu · 20 de marzo de 2026

  Artificial Intelligence (AI) systems are increasingly prominent in emerging smart cities, yet their reliability remains a critical concern. These systems typically operate through a sequence of interconnected functional stages, where upstream errors may propagate to downstream stages, ultimately aff…

• CoDA: Exploring Chain-of-Distribution Attacks and Post-Hoc Token-Space Repair for Medical Vision-Language Models

  Xiang Chen, Fangfang Yang, Chunlei Meng, Chengyin Hu, Ang Li, Yiwei Wei, Jiahuan Long, Jiujiang Guo · 20 de marzo de 2026

  Medical vision--language models (MVLMs) are increasingly used as perceptual backbones in radiology pipelines and as the visual front end of multimodal assistants, yet their reliability under real clinical workflows remains underexplored. Prior robustness evaluations often assume clean, curated input…

• Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test

  Xiaoyuan Zhu, Yaowen Ye, Tianyi Qiu, Hanlin Zhu, Sijun Tan, Ajraf Mannan, Jonathan Michala, Raluca Ada Popa, Willie Neiswanger · 20 de marzo de 2026

  As API access becomes a primary interface to large language models (LLMs), users often interact with black-box systems that offer little transparency into the deployed model. To reduce costs or maliciously alter model behaviors, API providers may discreetly serve quantized or fine-tuned variants, wh…

• On Optimizing Multimodal Jailbreaks for Spoken Language Models

  Aravind Krishnan, Karolina Sta\'nczak, Dietrich Klakow · 20 de marzo de 2026

  As Spoken Language Models (SLMs) integrate speech and text modalities, they inherit the safety vulnerabilities of their LLM backbone and an expanded attack surface. SLMs have been previously shown to be susceptible to jailbreaking, where adversarial prompts induce harmful responses. Yet existing att…

• FedTrident: Resilient Road Condition Classification Against Poisoning Attacks in Federated Learning

  Sheng Liu, Panos Papadimitratos · 20 de marzo de 2026

  FL has emerged as a transformative paradigm for ITS, notably camera-based Road Condition Classification (RCC). However, by enabling collaboration, FL-based RCC exposes the system to adversarial participants launching Targeted Label-Flipping Attacks (TLFAs). Malicious clients (vehicles) can relabel t…

• Rethinking Gradient-based Adversarial Attacks on Point Cloud Classification

  Jun Chen, Xinke Li, Mingyue Xu, Chongshou Li, Truiani Li · 20 de marzo de 2026

  Gradient-based adversarial attacks are widely used to evaluate the robustness of 3D point cloud classifiers, yet they often rely on uniform update rules that neglect point-wise heterogeneity, leading to perceptible perturbations. We propose two complementary strategies to improve both the effectiven…

• STEP: Detecting Audio Backdoor Attacks via Stability-based Trigger Exposure Profiling

  Kun Wang, Meng Chen, Junhao Wang, Yuli Wu, Li Lu, Chong Zhang, Peng Cheng, Jiaheng Zhang, Kui Ren · 20 de marzo de 2026

  With the widespread deployment of deep-learning-based speech models in security-critical applications, backdoor attacks have emerged as a serious threat: an adversary who poisons a small fraction of training data can implant a hidden trigger that controls the model's output while preserving normal b…

• Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference

  Pranay Anchuri, Matteo Campanelli, Paul Cesaretti, Rosario Gennaro, Tushar M. Jois, Hasan S. Kayman, Tugce Ozdemir · 20 de marzo de 2026

  When large AI models are deployed as cloud-based services, clients have no guarantee that responses are correct or were produced by the intended model. Rerunning inference locally is infeasible for large models, and existing cryptographic proof systems -- while providing strong correctness guarantee…

• Functional Subspace Watermarking for Large Language Models

  Zikang Ding, Junhao Li, Suling Wu, Junchi Yao, Hongbo Liu, Lijie Hu · 20 de marzo de 2026

  Model watermarking utilizes internal representations to protect the ownership of large language models (LLMs). However, these features inevitably undergo complex distortions during realistic model modifications such as fine-tuning, quantization, or knowledge distillation, making reliable extraction …

• VeriEquivBench: An Equivalence Score for Ground-Truth-Free Evaluation of Formally Verifiable Code

  Lingfei Zeng, Fengdi Che, Xuhan Huang, Fei Ye, Xu Xu, Binhang Yuan, Jie Fu · 20 de marzo de 2026

  Formal verification is the next frontier for ensuring the correctness of code generated by Large Language Models (LLMs). While methods that co-generate code and formal specifications in formal languages, like Dafny, can, in principle, prove alignment with user intent, progress is bottlenecked by spe…

• Causal Intervention Framework for Variational Auto Encoder Mechanistic Interpretability

  Dip Roy · 20 de marzo de 2026

  Mechanistic interpretability of deep learning models has emerged as a crucial research direction for understanding the functioning of neural networks. While significant progress has been made in interpreting discriminative models like transformers, understanding generative models such as Variational…

• MAED: Mathematical Activation Error Detection for Mitigating Physical Fault Attacks in DNN Inference

  Kasra Ahmadi, Saeed Aghapour, Mehran Mozaffari Kermani, Reza Azarderakhsh · 20 de marzo de 2026

  The inference phase of deep neural networks (DNNs) in embedded systems is increasingly vulnerable to fault attacks and failures, which can result in incorrect predictions. These vulnerabilities can potentially lead to catastrophic consequences, making the development of effective mitigation techniqu…

• rSDNet: Unified Robust Neural Learning against Label Noise and Adversarial Attacks

  Suryasis Jana, Abhik Ghosh · 19 de marzo de 2026

  Neural networks are central to modern artificial intelligence, yet their training remains highly sensitive to data contamination. Standard neural classifiers are trained by minimizing the categorical cross-entropy loss, corresponding to maximum likelihood estimation under a multinomial model. While …

• Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations

  Haozheng Luo, Yimin Wang, Jiahao Yu, Binghui Wang, Yan Chen · 19 de marzo de 2026

  We propose CRAFT, a red-teaming alignment framework that leverages model reasoning capabilities and hidden representations to improve robustness against jailbreak attacks. Unlike prior defenses that operate primarily at the output level, CRAFT aligns large reasoning models to generate safety-aware r…