Adversarial Robustness in Machine Learning

Neueste Paper

• Conditional Adversarial Fragility in Financial Machine Learning under Macroeconomic Stress

  Samruddhi Baviskar · 24. Dezember 2025

  Machine learning models used in financial decision systems operate in nonstationary economic environments, yet adversarial robustness is typically evaluated under static assumptions. This work introduces Conditional Adversarial Fragility, a regime dependent phenomenon in which adversarial vulnerabil…

• Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning

  Jiayun Wu, Jiashuo Liu, Zhiyuan Zeng, Tianyang Zhan, Wenhao Huang · 24. Dezember 2025

  LLM deployment in critical domains is currently impeded by persistent hallucinations--generating plausible but factually incorrect assertions. While scaling laws drove significant improvements in general capabilities, theoretical frameworks suggest hallucination is not merely stochastic error but a …

• Membership Inference Attack with Partial Features

  Xurun Wang, Guangrui Liu, Xinjie Li, Haoyu He, Lin Yao, Zhongyun Hua, Weizhe Zhang · 24. Dezember 2025

  Machine learning models are vulnerable to membership inference attack, which can be used to determine whether a given sample appears in the training data. Most existing methods assume the attacker has full access to the features of the target sample. This assumption, however, does not hold in many r…

• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications

  Honglin Mu, Jinghao Liu, Kaiyang Wan, Rui Xing, Xiuying Chen, Timothy Baldwin, Wanxiang Che · 24. Dezember 2025

  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation. However, our research identifies a vulnerability: LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or cod…

• PHANTOM: PHysical ANamorphic Threats Obstructing Connected Vehicle Mobility

  Md Nahid Hasan Shuvo, Moinul Hossain · 24. Dezember 2025

  Connected autonomous vehicles (CAVs) rely on vision-based deep neural networks (DNNs) and low-latency (Vehicle-to-Everything) V2X communication to navigate safely and efficiently. Despite their advances, these systems remain vulnerable to physical adversarial attacks. In this paper, we introduce PHA…

• ArcGen: Generalizing Neural Backdoor Detection Across Diverse Architectures

  Zhonghao Yang, Cheng Luo, Daojing He, Yiming Li, Yu Li · 24. Dezember 2025

  Backdoor attacks pose a significant threat to the security and reliability of deep learning models. To mitigate such attacks, one promising approach is to learn to extract features from the target model and use these features for backdoor detection. However, we discover that existing learning-based …

• Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography

  Songze Li, Jiameng Cheng, Yiming Li, Xiaojun Jia, Dacheng Tao · 24. Dezember 2025

  By integrating language understanding with perceptual modalities such as images, multimodal large language models (MLLMs) constitute a critical substrate for modern AI systems, particularly intelligent agents operating in open and interactive environments. However, their increasing accessibility als…

• Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassifications

  Lorenzo Capelli, Leandro de Souza Rosa, Gianluca Setti, Mauro Mangia, Riccardo Rovatti · 23. Dezember 2025

  The recent explosive growth in Deep Neural Networks applications raises concerns about the black-box usage of such models, with limited trasparency and trustworthiness in high-stakes domains, which have been crystallized as regulatory requirements such as the European Union Artificial Intelligence A…

• Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models

  Wei Qian, Chenxu Zhao, Yangyi Li, Mengdi Huai · 23. Dezember 2025

  The rapid advancements in artificial intelligence (AI) have primarily focused on the process of learning from data to acquire knowledgeable learning systems. As these systems are increasingly deployed in critical areas, ensuring their privacy and alignment with human values is paramount. Recently, s…

• Efficient Jailbreak Mitigation Using Semantic Linear Classification in a Multi-Staged Pipeline

  Akshaj Prashanth Rao, Advait Singh, Saumya Kumaar Saksena, Dhruv Kumar · 23. Dezember 2025

  Prompt injection and jailbreaking attacks pose persistent security challenges to large language model (LLM)-based systems. We present an efficient and systematically evaluated defense architecture that mitigates these threats through a lightweight, multi-stage pipeline. Its core component is a seman…

• TAPAS: Datasets for Learning the Learning with Errors Problem

  Eshika Saxena, Alberto Alfarano, Fran\c{c}ois Charton, Emily Wenger, Kristin Lauter · 23. Dezember 2025

  AI-powered attacks on Learning with Errors (LWE), an important hard math problem in post-quantum cryptography, rival or outperform "classical" attacks on LWE under certain parameter settings. Despite the promise of this approach, a dearth of accessible data limits AI practitioners' ability to study …

• A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage

  Francis Bach (SIERRA) · 23. Dezember 2025

  We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lov{\'a}sz extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss fu…

• The 6th International Verification of Neural Networks Competition (VNN-COMP 2025): Summary and Results

  Konstantin Kaulen, Tobias Ladner, Stanley Bak, Christopher Brix, Hai Duong, Thomas Flinkow, Taylor T. Johnson, Lukas Koller, Edoardo Manino, ThanhVu H Nguyen, Haoze Wu · 23. Dezember 2025

  This report summarizes the 6th International Verification of Neural Networks Competition (VNN-COMP 2025), held as a part of the 8th International Symposium on AI Verification (SAIV), that was collocated with the 37th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held ann…

• SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples

  Haoye Lu, Yaoliang Yu, Darren Ho · 22. Dezember 2025

  In many real-world scenarios, obtaining fully observed samples is prohibitively expensive or even infeasible, while partial and noisy observations are comparatively easy to collect. In this work, we study distribution restoration with abundant noisy samples, assuming the corruption process is availa…

• Adversarial VR: An Open-Source Testbed for Evaluating Adversarial Robustness of VR Cybersickness Detection and Mitigation

  Istiak Ahmed, Ripan Kumar Kundu, Khaza Anuarul Hoque · 22. Dezember 2025

  Deep learning (DL)-based automated cybersickness detection methods, along with adaptive mitigation techniques, can enhance user comfort and interaction. However, recent studies show that these DL-based systems are susceptible to adversarial attacks; small perturbations to sensor inputs can degrade m…

• Holmes: Towards Effective and Harmless Model Ownership Verification to Personalized Large Vision Models via Decoupling Common Features

  Linghui Zhu, Yiming Li, Haiqin Weng, Yan Liu, Tianwei Zhang, Shu-Tao Xia, Zhi Wang · 22. Dezember 2025

  Large vision models (LVMs) achieve remarkable performance in various downstream tasks, primarily by personalizing pre-trained models through fine-tuning with private and valuable local data, which makes the personalized model a valuable intellectual property. Similar to the era of traditional DNNs, …

• Verifiability-First Agents: Provable Observability and Lightweight Audit Agents for Controlling Autonomous LLM Systems

  Abhivansh Gupta · 22. Dezember 2025

  As LLM-based agents grow more autonomous and multi-modal, ensuring they remain controllable, auditable, and faithful to deployer intent becomes critical. Prior benchmarks measured the propensity for misaligned behavior and showed that agent personalities and tool access significantly influence misal…

• Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks

  Xuancun Lu, Jiaxiang Chen, Shilin Xiao, Zizhi Jin, Zhangrui Chen, Hanwen Yu, Bohan Qian, Ruochen Zhou, Xiaoyu Ji, Wenyuan Xu · 22. Dezember 2025

  Vision-Language-Action (VLA) models revolutionize robotic systems by enabling end-to-end perception-to-action pipelines that integrate multiple sensory modalities, such as visual signals processed by cameras and auditory signals captured by microphones. This multi-modality integration allows VLA mod…

• MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval

  Saksham Sahai Srivastava, Haoyu He · 22. Dezember 2025

  Large Language Model (LLM) agents increasingly rely on long-term memory and Retrieval-Augmented Generation (RAG) to persist experiences and refine future performance. While this experience learning capability enhances agentic autonomy, it introduces a critical, unexplored attack surface, i.e., the t…

• Adversarial Robustness of Vision in Open Foundation Models

  Jonathon Fox, William J Buchanan, Pavlos Papadopoulos · 22. Dezember 2025

  With the increase in deep learning, it becomes increasingly difficult to understand the model in which AI systems can identify objects. Thus, an adversary could aim to modify an image by adding unseen elements, which will confuse the AI in its recognition of an entity. This paper thus investigates t…

• QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systems

  Yiliu Yang, Yilei Jiang, Qunzhong Wang, Yingshui Tan, Xiaoyong Zhu, Sherman S. M. Chow, Bo Zheng, Xiangyu Yue · 19. Dezember 2025

  Safety risks arise as large language model-based agents solve complex tasks with tools, multi-step plans, and inter-agent messages. However, deployer-written policies in natural language are ambiguous and context dependent, so they map poorly to machine-checkable rules, and runtime enforcement is un…

• Design and Evaluation of Cost-Aware PoQ for Decentralized LLM Inference

  Arther Tian, Alex Ding, Frank Chen, Alan Wu, Aaron Chan, Bruce Zhang · 19. Dezember 2025

  Decentralized large language model (LLM) inference promises transparent and censorship resistant access to advanced AI, yet existing verification approaches struggle to scale to modern models. Proof of Quality (PoQ) replaces cryptographic verification of computation with consensus over output qualit…

• Autoencoder-based Denoising Defense against Adversarial Attacks on Object Detection

  Min Geun Song, Gang Min Kim, Woonmin Kim, Yongsik Kim, Jeonghyun Sim, Sangbeom Park, Huy Kang Kim · 19. Dezember 2025

  Deep learning-based object detection models play a critical role in real-world applications such as autonomous driving and security surveillance systems, yet they remain vulnerable to adversarial examples. In this work, we propose an autoencoder-based denoising defense to recover object detection pe…

• PediatricAnxietyBench: Evaluating Large Language Model Safety Under Parental Anxiety and Pressure in Pediatric Consultations

  Vahideh Zolfaghari · 19. Dezember 2025

  Large language models (LLMs) are increasingly consulted by parents for pediatric guidance, yet their safety under real-world adversarial pressures is poorly understood. Anxious parents often use urgent language that can compromise model safeguards, potentially causing harmful advice. PediatricAnxiet…

• Feature-Selective Representation Misdirection for Machine Unlearning

  Taozhao Chen, Linghan Huang, Kim-Kwang Raymond Choo, Huaming Chen · 19. Dezember 2025

  As large language models (LLMs) are increasingly adopted in safety-critical and regulated sectors, the retention of sensitive or prohibited knowledge introduces escalating risks, ranging from privacy leakage to regulatory non-compliance to to potential misuse, and so on. Recent studies suggest that …