Adversarial Robustness in Machine Learning

Últimos artículos

• How a Bit Becomes a Story: Semantic Steering via Differentiable Fault Injection

  Zafaryab Haider, Md Hafizur Rahman, Shane Moeykens, Vijay Devabhaktuni, Prabuddha Chakraborty · 18 de diciembre de 2025

  Hard-to-detect hardware bit flips, from either malicious circuitry or bugs, have already been shown to make transformers vulnerable in non-generative tasks. This work, for the first time, investigates how low-level, bitwise perturbations (fault injection) to the weights of a large language model (LL…

• The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systems

  Debu Sinha · 18 de diciembre de 2025

  Retrieval-Augmented Generation (RAG) systems remain susceptible to hallucinations despite grounding in retrieved evidence. Current detection methods rely on semantic similarity and natural language inference (NLI), but their fundamental limitations have not been rigorously characterized. We apply co…

• Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance

  Ioannis Kalogeropoulos, Giorgos Bouritsas, Yannis Panagakis · 18 de diciembre de 2025

  As machine learning models are increasingly deployed in high-stakes settings, e.g. as decision support systems in various societal sectors or in critical infrastructure, designers and auditors are facing the need to ensure that models satisfy a wider variety of requirements (e.g. compliance with reg…

• Persistent Backdoor Attacks under Continual Fine-Tuning of LLMs

  Jing Cui, Yufei Han, Jianbin Jiao, Junge Zhang · 18 de diciembre de 2025

  Backdoor attacks embed malicious behaviors into Large Language Models (LLMs), enabling adversaries to trigger harmful outputs or bypass safety controls. However, the persistence of the implanted backdoors under user-driven post-deployment continual fine-tuning has been rarely examined. Most prior wo…

• CAPE: Capability Achievement via Policy Execution

  David Ball · 18 de diciembre de 2025

  Modern AI systems lack a way to express and enforce requirements. Pre-training produces intelligence, and post-training optimizes preferences, but neither guarantees that models reliably satisfy explicit, context-dependent constraints. This missing abstraction explains why highly intelligent models …

• SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxification

  Hongbo Wang, MaungMaung AprilPyone, Isao Echizen · 18 de diciembre de 2025

  Disclaimer: Samples in this paper may be harmful and cause discomfort. Multimodal large language models (MLLMs) enable multimodal generation but inherit toxic, biased, and NSFW signals from weakly curated pretraining corpora, causing safety risks, especially under adversarial triggers that late, o…

• MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers

  Xuanjun Zong, Zhiqi Shen, Lei Wang, Yunshi Lan, Chao Yang · 18 de diciembre de 2025

  Large language models (LLMs) are evolving into agentic systems that reason, plan, and operate external tools. The Model Context Protocol (MCP) is a key enabler of this transition, offering a standardized interface for connecting LLMs with heterogeneous tools and services. Yet MCP's openness and mult…

• Bits for Privacy: Evaluating Post-Training Quantization via Membership Inference

  Chenxiang Zhang, Tongxi Qu, Zhong Li, Tian Zhang, Jun Pang, Sjouke Mauw · 18 de diciembre de 2025

  Deep neural networks are widely deployed with quantization techniques to reduce memory and computational costs by lowering the numerical precision of their parameters. While quantization alters model parameters and their outputs, existing privacy analyses primarily focus on full-precision models, le…

• Stylized Synthetic Augmentation further improves Corruption Robustness

  Georg Siedel, Rojan Regmi, Abhirami Anand, Weijia Shao, Silvia Vock, Andrey Morozov · 18 de diciembre de 2025

  This paper proposes a training data augmentation pipeline that combines synthetic image data with neural style transfer in order to address the vulnerability of deep vision models to common corruptions. We show that although applying style transfer on synthetic images degrades their quality with res…

• Entropy-Reservoir Bregman Projection: An Information-Geometric Unification of Model Collapse

  Jingwei Chen · 18 de diciembre de 2025

  Self-referential learning -- training a model on data it generated itself -- promises boundless scalability but chronically suffers from model collapse: language models degenerate into repetitive text, GANs drop modes, and reinforcement-learning policies over-exploit. Although practitioners employ a…

• Factor(U,T): Controlling Untrusted AI by Monitoring their Plans

  Edward Lue Chee Lip, Anthony Channg, Diana Kim, Aaron Sandoval, Kevin Zhu · 18 de diciembre de 2025

  As AI capabilities advance, we increasingly rely on powerful models to decompose complex tasks $\unicode{x2013}$ but what if the decomposer itself is malicious? Factored cognition protocols decompose complex tasks into simpler child tasks: one model creates the decomposition, while other models impl…

• One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs

  Yixin Tan, Zhe Yu, Jun Sakuma · 18 de diciembre de 2025

  Finetuning pretrained large language models (LLMs) has become the standard paradigm for developing downstream applications. However, its security implications remain unclear, particularly regarding whether finetuned LLMs inherit jailbreak vulnerabilities from their pretrained sources. We investigate…

• Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems

  Kamel Kamel, Hridoy Sankar Dutta, Keshav Sood, Sunil Aryal · 18 de diciembre de 2025

  Voice Authentication Systems (VAS) use unique vocal characteristics for verification. They are increasingly integrated into high-security sectors such as banking and healthcare. Despite their improvements using deep learning, they face severe vulnerabilities from sophisticated threats like deepfakes…

• Imitation Game: Reproducing Deep Learning Bugs Leveraging an Intelligent Agent

  Mehil B Shah, Mohammad Masudur Rahman, Foutse Khomh · 18 de diciembre de 2025

  Despite their wide adoption in various domains (e.g., healthcare, finance, software engineering), Deep Learning (DL)-based applications suffer from many bugs, failures, and vulnerabilities. Reproducing these bugs is essential for their resolution, but it is extremely challenging due to the inherent …

• Quantum-Augmented AI/ML for O-RAN: Hierarchical Threat Detection with Synergistic Intelligence and Interpretability (Technical Report)

  Tan Le, Van Le, Sachin Shetty · 18 de diciembre de 2025

  Open Radio Access Networks (O-RAN) enhance modularity and telemetry granularity but also widen the cybersecurity attack surface across disaggregated control, user and management planes. We propose a hierarchical defense framework with three coordinated layers-anomaly detection, intrusion confirmatio…

• Quantifying Return on Security Controls in LLM Systems

  Richard Helder Moulton, Austin O'Brien, John D. Hastings · 18 de diciembre de 2025

  Although large language models (LLMs) are increasingly used in security-critical workflows, practitioners lack quantitative guidance on which safeguards are worth deploying. This paper introduces a decision-oriented framework and reproducible methodology that together quantify residual risk, convert…

• BashArena: A Control Setting for Highly Privileged AI Agents

  Adam Kaufman, James Lucassen, Tyler Tracy, Cody Rushing, Aryan Bhatt · 18 de diciembre de 2025

  Future AI agents might run autonomously with elevated privileges. If these agents are misaligned, they might abuse these privileges to cause serious damage. The field of AI control develops techniques that make it harder for misaligned AIs to cause such damage, while preserving their usefulness. We …

• Benchmarking Gaslighting Negation Attacks Against Reasoning Models

  Bin Zhu, Hailong Yin, Jingjing Chen, Yu-Gang Jiang · 18 de diciembre de 2025

  Recent advances in reasoning-centric models promise improved robustness through mechanisms such as chain-of-thought prompting and test-time scaling. However, their ability to withstand gaslighting negation attacks-adversarial prompts that confidently deny correct answers-remains underexplored. In th…

• A Critical Perspective on Finite Sample Conformal Prediction Theory in Medical Applications

  Klaus-Rudolf Kladny, Bernhard Sch\"olkopf, Lisa Koch, Christian F. Baumgartner, Michael Muehlebach · 18 de diciembre de 2025

  Machine learning (ML) is transforming healthcare, but safe clinical decisions demand reliable uncertainty estimates that standard ML models fail to provide. Conformal prediction (CP) is a popular tool that allows users to turn heuristic uncertainty estimates into uncertainty estimates with statistic…

• Remotely Detectable Robot Policy Watermarking

  Michael Amir, Manon Flageat, Amanda Prorok · 18 de diciembre de 2025

  The success of machine learning for real-world robotic systems has created a new form of intellectual property: the trained policy. This raises a critical need for novel methods that verify ownership and detect unauthorized, possibly unsafe misuse. While watermarking is established in other domains,…

• Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis

  Srishti Gupta, Zhang Chen, Luca Demetrio, Xiaoyi Feng, Zhaoqiang Xia, Antonio Emanuele Cin\`a, Maura Pintor, Luca Oneto, Ambra Demontis, Battista Biggio, Fabio Roli · 18 de diciembre de 2025

  Thanks to their extensive capacity, over-parameterized neural networks exhibit superior predictive capabilities and generalization. However, having a large parameter space is considered one of the main suspects of the neural networks' vulnerability to adversarial example -- input samples crafted ad-…

• TrajSyn: Privacy-Preserving Dataset Distillation from Federated Model Trajectories for Server-Side Adversarial Training

  Mukur Gupta, Niharika Gupta, Saifur Rahman, Shantanu Pal, Chandan Karmakar · 18 de diciembre de 2025

  Deep learning models deployed on edge devices are increasingly used in safety-critical applications. However, their vulnerability to adversarial perturbations poses significant risks, especially in Federated Learning (FL) settings where identical models are distributed across thousands of clients. W…

• Weighted Conformal Prediction Provides Adaptive and Valid Mask-Conditional Coverage for General Missing Data Mechanisms

  Jiarong Fan, Juhyun Park. Thi Phuong Thuy Vo, Nicolas Brunel · 17 de diciembre de 2025

  Conformal prediction (CP) offers a principled framework for uncertainty quantification, but it fails to guarantee coverage when faced with missing covariates. In addressing the heterogeneity induced by various missing patterns, Mask-Conditional Valid (MCV) Coverage has emerged as a more desirable pr…

• On the Hardness of Conditional Independence Testing In Practice

  Zheng He, Roman Pogodin, Yazhe Li, Namrata Deka, Arthur Gretton, Danica J. Sutherland · 17 de diciembre de 2025

  Tests of conditional independence (CI) underpin a number of important problems in machine learning and statistics, from causal discovery to evaluation of predictor fairness and out-of-distribution robustness. Shah and Peters (2020) showed that, contrary to the unconditional case, no universally fini…

• Reliable Statistical Guarantees for Conformal Predictors with Small Datasets

  Miguel S\'anchez-Dom\'inguez, Lucas Lacasa, Javier de Vicente, Gonzalo Rubio, Eusebio Valero · 17 de diciembre de 2025

  Surrogate models (including deep neural networks and other machine learning algorithms in supervised learning) are capable of approximating arbitrarily complex, high-dimensional input-output problems in science and engineering, but require a thorough data-agnostic uncertainty quantification analysis…