Adversarial Robustness in Machine Learning

Neueste Paper

• Zero-Knowledge Proof Based Verifiable Inference of Models

  Yunxiao Wang · 26. November 2025

  Recent advances in artificial intelligence (AI), particularly deep learning, have led to widespread adoption across various applications. Yet, a fundamental challenge persists: how can we verify the correctness of AI model inference when model owners cannot (or will not) reveal their parameters? The…

• On the Feasibility of Hijacking MLLMs' Decision Chain via One Perturbation

  Changyue Li, Jiaying Li, Youliang Yuan, Jiaming He, Zhicong Huang, Pinjia He · 26. November 2025

  Conventional adversarial attacks focus on manipulating a single decision of neural networks. However, real-world models often operate in a sequence of decisions, where an isolated mistake can be easily corrected, but cascading errors can lead to severe risks. This paper reveals a novel threat: a s…

• SafeFix: Targeted Model Repair via Controlled Image Generation

  Ouyang Xu, Baoming Zhang, Ruiyu Mao, Yunhui Guo · 26. November 2025

  Deep learning models for visual recognition often exhibit systematic errors due to underrepresented semantic subpopulations. Although existing debugging frameworks can pinpoint these failures by identifying key failure attributes, repairing the model effectively remains difficult. Current solutions …

• Shortcut Invariance: Targeted Jacobian Regularization in Disentangled Latent Space

  Shivam Pal, Sakshi Varshney, Piyush Rai · 26. November 2025

  Deep neural networks are prone to learning shortcuts, spurious and easily learned correlations in training data that cause severe failures in out-of-distribution (OOD) generalization. A dominant line of work seeks robustness by learning a robust representation, often explicitly partitioning the late…

• stable-pretraining-v1: Foundation Model Research Made Simple

  Randall Balestriero, Hugues Van Assel, Sami BuGhanem, Lucas Maes · 26. November 2025

  Foundation models and self-supervised learning (SSL) have become central to modern AI, yet research in this area remains hindered by complex codebases, redundant re-implementations, and the heavy engineering burden of scaling experiments. We present stable-pretraining, a modular, extensible, and per…

• DiFR: Inference Verification Despite Nondeterminism

  Adam Karvonen, Daniel Reuter, Roy Rinberg, Luke Marks, Adri\`a Garriga-Alonso, Keri Warr · 26. November 2025

  As demand for LLM inference grows, it is becoming increasingly important that providers and their customers can verify that inference processes are performed correctly, without errors or tampering. However, re-running the same inference process twice often leads to different results due to benign nu…

• An Invariant Latent Space Perspective on Language Model Inversion

  Wentao Ye, Jiaqi Hu, Haobo Wang, Xinpeng Ti, Zhiqing Xiao, Hao Chen, Liyao Li, Lei Feng, Sai Wu, Junbo Zhao · 26. November 2025

  Language model inversion (LMI), i.e., recovering hidden prompts from outputs, emerges as a concrete threat to user privacy and system security. We recast LMI as reusing the LLM's own latent space and propose the Invariant Latent Space Hypothesis (ILSH): (1) diverse outputs from the same source promp…

• ARBoids: Adaptive Residual Reinforcement Learning With Boids Model for Cooperative Multi-USV Target Defense

  Jiyue Tao, Tongsheng Shen, Dexin Zhao, Feitian Zhang · 26. November 2025

  The target defense problem (TDP) for unmanned surface vehicles (USVs) concerns intercepting an adversarial USV before it breaches a designated target region, using one or more defending USVs. A particularly challenging scenario arises when the attacker exhibits superior maneuverability compared to t…

• Prompt Fencing: A Cryptographic Approach to Establishing Security Boundaries in Large Language Model Prompts

  Steven Peh · 26. November 2025

  Large Language Models (LLMs) remain vulnerable to prompt injection attacks, representing the most significant security threat in production deployments. We present Prompt Fencing, a novel architectural approach that applies cryptographic authentication and data architecture principles to establish e…

• Cross-LLM Generalization of Behavioral Backdoor Detection in AI Agent Supply Chains

  Arun Chowdary Sanna · 26. November 2025

  As AI agents become integral to enterprise workflows, their reliance on shared tool libraries and pre-trained components creates significant supply chain vulnerabilities. While previous work has demonstrated behavioral backdoor detection within individual LLM architectures, the critical question of …

• Towards Trustworthy Wi-Fi Sensing: Systematic Evaluation of Deep Learning Model Robustness to Adversarial Attacks

  Shreevanth Krishnaa Gopalakrishnan, Stephen Hailes · 26. November 2025

  Machine learning has become integral to Channel State Information (CSI)-based human sensing systems and is expected to power applications such as device-free activity recognition and identity detection in future cellular and Wi-Fi generations. However, these systems rely on models whose decisions ca…

• Exploiting the Experts: Unauthorized Compression in MoE-LLMs

  Pinaki Prasad Guha Neogi, Ahmad Mohammadshirazi, Dheeraj Kulshrestha, Rajiv Ramnath · 26. November 2025

  Mixture-of-Experts (MoE) architectures are increasingly adopted in large language models (LLMs) for their scalability and efficiency. However, their modular structure introduces a unique vulnerability: adversaries can attempt to compress or repurpose models by pruning experts and cheaply fine-tuning…

• SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data

  Penghao Rao, Runmin Jiang, Min Xu · 26. November 2025

  Approximating training-point influence on test predictions is critical for deploying deep-learning vision models, essential for locating noisy data. Though the influence function was proposed for attributing how infinitesimal up-weighting or removal of individual training examples affects model outp…

• Fighting AI with AI: Leveraging Foundation Models for Assuring AI-Enabled Safety-Critical Systems

  Anastasia Mavridou, Divya Gopinath, Corina S. P\u{a}s\u{a}reanu · 26. November 2025

  The integration of AI components, particularly Deep Neural Networks (DNNs), into safety-critical systems such as aerospace and autonomous vehicles presents fundamental challenges for assurance. The opacity of AI systems, combined with the semantic gap between high-level requirements and low-level ne…

• PaTAS: A Parallel System for Trust Propagation in Neural Networks Using Subjective Logic

  Koffi Ismael Ouattara, Ioannis Krontiris, Theo Dimitrakos, Dennis Eisermann, Frank Kargl · 26. November 2025

  Trustworthiness has become a key requirement for the deployment of artificial intelligence systems in safety-critical applications. Conventional evaluation metrics such as accuracy and precision fail to capture uncertainty or the reliability of model predictions, particularly under adversarial or de…

• From One Attack Domain to Another: Contrastive Transfer Learning with Siamese Networks for APT Detection

  Sidahmed Benabderrahmane, Talal Rahwan · 26. November 2025

  Advanced Persistent Threats (APT) pose a major cybersecurity challenge due to their stealth, persistence, and adaptability. Traditional machine learning detectors struggle with class imbalance, high dimensional features, and scarce real world traces. They often lack transferability-performing well i…

• Boundary on the Table: Efficient Black-Box Decision-Based Attacks for Structured Data

  Roie Kazoom, Yuval Ratzabi, Etamar Rothstein, Ofer Hadar · 25. November 2025

  Adversarial robustness in structured data remains an underexplored frontier compared to vision and language domains. In this work, we introduce a novel black-box, decision-based adversarial attack tailored for tabular data. Our approach combines gradient-free direction estimation with an iterative b…

• The Horcrux: Mechanistically Interpretable Task Decomposition for Detecting and Mitigating Reward Hacking in Embodied AI Systems

  Subramanyam Sahoo, Jared Junkin · 25. November 2025

  Embodied AI agents exploit reward signal flaws through reward hacking, achieving high proxy scores while failing true objectives. We introduce Mechanistically Interpretable Task Decomposition (MITD), a hierarchical transformer architecture with Planner, Coordinator, and Executor modules that detects…

• Vulnerability-Aware Robust Multimodal Adversarial Training

  Junrui Zhang, Xinyu Zhao, Jie Peng, Chenjie Wang, Jianmin Ji, Tianlong Chen · 25. November 2025

  Multimodal learning has shown significant superiority on various tasks by integrating multiple modalities. However, the interdependencies among modalities increase the susceptibility of multimodal models to adversarial attacks. Existing methods mainly focus on attacks on specific modalities or indis…

• Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic

  Mostafa Mozafari, Farooq Ahmad Wani, Maria Sofia Bucarelli, Fabrizio Silvestri · 25. November 2025

  Corrupted training data are ubiquitous. Corrective Machine Unlearning (CMU) seeks to remove the influence of such corruption post-training. Prior CMU typically assumes access to identified corrupted training samples (a ``forget set''). However, in many real-world scenarios the training data are no l…

• Towards Realistic Guarantees: A Probabilistic Certificate for SmoothLLM

  Adarsh Kumarappan, Ayushi Mehrotra · 25. November 2025

  The SmoothLLM defense provides a certification guarantee against jailbreaking attacks, but it relies on a strict `k-unstable' assumption that rarely holds in practice. This strong assumption can limit the trustworthiness of the provided safety certificate. In this work, we address this limitation by…

• Curvature-Aware Safety Restoration In LLMs Fine-Tuning

  Thong Bach, Thanh Nguyen-Tang, Dung Nguyen, Thao Minh Le, Truyen Tran · 25. November 2025

  Fine-tuning Large Language Models (LLMs) for downstream tasks often compromises safety alignment, even when using parameter-efficient methods like LoRA. In this work, we uncover a notable property: fine-tuned models preserve the geometric structure of their loss landscapes concerning harmful content…

• Targeted Manipulation: Slope-Based Attacks on Financial Time-Series Data

  Dominik Luszczynski · 25. November 2025

  A common method of attacking deep learning models is through adversarial attacks, which occur when an attacker specifically modifies the input of a model to produce an incorrect result. Adversarial attacks have been deeply investigated in the image domain; however, there is less research in the time…

• Ensuring Calibration Robustness in Split Conformal Prediction Under Adversarial Attacks

  Xunlei Qian, Yue Xing · 25. November 2025

  Conformal prediction (CP) provides distribution-free, finite-sample coverage guarantees but critically relies on exchangeability, a condition often violated under distribution shift. We study the robustness of split conformal prediction under adversarial perturbations at test time, focusing on both …

• How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints

  Kensuke Nakamura, Arun L. Bishop, Steven Man, Aaron M. Johnson, Zachary Manchester, Andrea Bajcsy · 25. November 2025

  Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling safe visuomotor control under hard-to-model constraints. However, existing methods implement "least-restrictive" filter…