Adversarial Robustness in Machine Learning

Neueste Paper

• Optimizing In-Context Learning for Efficient Full Conformal Prediction

  Weicao Deng, Sangwoo Park, Min Li, Osvaldo Simeone · 20. November 2025

  Reliable uncertainty quantification is critical for trustworthy AI. Conformal Prediction (CP) provides prediction sets with distribution-free coverage guarantees, but its two main variants face complementary limitations. Split CP (SCP) suffers from data inefficiency due to dataset partitioning, whil…

• VeriFlow: Modeling Distributions for Neural Network Verification

  Faried Abu Zaid, Daniel Neider, Mustafa Yal\c{c}{\i}ner · 20. November 2025

  Formal verification has emerged as a promising method to ensure the safety and reliability of neural networks. However, many relevant properties, such as fairness or global robustness, pertain to the entire input space. If one applies verification techniques naively, the neural network is checked ev…

• Attacking Autonomous Driving Agents with Adversarial Machine Learning: A Holistic Evaluation with the CARLA Leaderboard

  Henry Wong, Clement Fung, Weiran Lin, Karen Li, Stanley Chen, Lujo Bauer · 20. November 2025

  To autonomously control vehicles, driving agents use outputs from a combination of machine-learning (ML) models, controller logic, and custom modules. Although numerous prior works have shown that adversarial examples can mislead ML models used in autonomous driving contexts, it remains unclear if t…

• Transformer Injectivity & Geometric Robustness - Analytic Margins and Bi-Lipschitz Uniformity of Sequence-Level Hidden States

  Mikael von Strauss · 20. November 2025

  Under real-analytic assumptions on decoder-only Transformers, recent work shows that the map from discrete prompts to last-token hidden states is generically injective on finite prompt sets. We refine this picture: for each layer $\ell$ we define a collision discriminant $\Delta^\ell \subset \Theta$…

• Preference Learning with Lie Detectors can Induce Honesty or Evasion

  Chris Cundy, Adam Gleave · 19. November 2025

  As AI systems become more capable, deceptive behaviors can undermine evaluation and mislead users at deployment. Recent work has shown that lie detectors can accurately classify deceptive behavior, but they are not typically used in the training pipeline due to concerns around contamination and obje…

• Sharp detection of low-dimensional structure in probability measures via dimensional logarithmic Sobolev inequalities

  Matthew T. C. Li, Tiangang Cui, Fengyi Li, Youssef Marzouk, Olivier Zahm · 19. November 2025

  Identifying low-dimensional structure in high-dimensional probability measures is an essential pre-processing step for efficient sampling. We introduce a method for identifying and approximating a target measure $\pi$ as a perturbation of a given reference measure $\mu$ along a few significant direc…

• Empirical Likelihood for Random Forests and Ensembles

  Harold D. Chiang, Yukitoshi Matsushita, Taisuke Otsu · 19. November 2025

  We develop an empirical likelihood (EL) framework for random forests and related ensemble methods, providing a likelihood-based approach to quantify their statistical uncertainty. Exploiting the incomplete $U$-statistic structure inherent in ensemble predictions, we construct an EL statistic that is…

• Dynamic Black-box Backdoor Attacks on IoT Sensory Data

  Ajesh Koyatan Chathoth, Stephen Lee · 19. November 2025

  Sensor data-based recognition systems are widely used in various applications, such as gait-based authentication and human activity recognition (HAR). Modern wearable and smart devices feature various built-in Inertial Measurement Unit (IMU) sensors, and such sensor-based measurements can be fed to …

• Watch Out for the Lifespan: Evaluating Backdoor Attacks Against Federated Model Adaptation

  Bastien Vuillod, Pierre-Alain Moellic, Jean-Max Dutertre · 19. November 2025

  Large models adaptation through Federated Learning (FL) addresses a wide range of use cases and is enabled by Parameter-Efficient Fine-Tuning techniques such as Low-Rank Adaptation (LoRA). However, this distributed learning paradigm faces several security threats, particularly to its integrity, such…

• AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection

  Peiran Wang, Yang Liu, Yunfei Lu, Yifeng Cai, Hongbo Chen, Qingyou Yang, Jie Zhang, Jue Hong, Ye Wu · 19. November 2025

  Large Language Model (LLM) agents offer a powerful new paradigm for solving various problems by combining natural language reasoning with the execution of external tools. However, their dynamic and non-transparent behavior introduces critical security risks, particularly in the presence of prompt in…

• TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning

  Mingxuan Zhang, Oubo Ma, Kang Wei, Songze Li, Shouling Ji · 19. November 2025

  Deep reinforcement learning (DRL) has achieved remarkable success in a wide range of sequential decision-making applications, including robotics, healthcare, smart grids, and finance. Recent studies reveal that adversaries can implant backdoors into DRL agents during the training phase. These backdo…

• Intervention Efficiency and Perturbation Validation Framework: Capacity-Aware and Robust Clinical Model Selection under the Rashomon Effect

  Yuwen Zhang, Viet Tran, Paul Weng · 19. November 2025

  In clinical machine learning, the coexistence of multiple models with comparable performance -- a manifestation of the Rashomon Effect -- poses fundamental challenges for trustworthy deployment and evaluation. Small, imbalanced, and noisy datasets, coupled with high-dimensional and weakly identified…

• Comparing Task-Agnostic Embedding Models for Tabular Data

  Frederik Hoppe, Lars Kleinemeier, Astrid Franz, Udo G\"obel · 19. November 2025

  Recent foundation models for tabular data achieve strong task-specific performance via in-context learning. Nevertheless, they focus on direct prediction by encapsulating both representation learning and task-specific inference inside a single, resource-intensive network. This work specifically focu…

• N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluator

  Zheyu Lin, Jirui Yang, Hengqi Guo, Yubing Bao, Yao Guan · 19. November 2025

  Evaluating the safety robustness of LLMs is critical for their deployment. However, mainstream Red Teaming methods rely on online generation and black-box output analysis. These approaches are not only costly but also suffer from feedback latency, making them unsuitable for agile diagnostics after t…

• Credal Ensemble Distillation for Uncertainty Quantification

  Kaizheng Wang, Fabio Cuzzolin, David Moens, Hans Hallez · 19. November 2025

  Deep ensembles (DE) have emerged as a powerful approach for quantifying predictive uncertainty and distinguishing its aleatoric and epistemic components, thereby enhancing model robustness and reliability. However, their high computational and memory costs during inference pose significant challenge…

• Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments

  Samuel Nathanson, Rebecca Williams, Cynthia Matuszek · 19. November 2025

  Large language models (LLMs) increasingly operate in multi-agent and safety-critical settings, raising open questions about how their vulnerabilities scale when models interact adversarially. This study examines whether larger models can systematically jailbreak smaller ones - eliciting harmful or r…

• Certified but Fooled! Breaking Certified Defences with Ghost Certificates

  Quoc Viet Vo, Tashreque M. Haq, Paul Montague, Tamas Abraham, Ehsan Abbasnejad, Damith C. Ranasinghe · 19. November 2025

  Certified defenses promise provable robustness guarantees. We study the malicious exploitation of probabilistic certification frameworks to better understand the limits of guarantee provisions. Now, the objective is to not only mislead a classifier, but also manipulate the certification process to g…

• Robustness of LLM-enabled vehicle trajectory prediction under data security threats

  Feilong Wang, Fuqiang Liu · 19. November 2025

  The integration of large language models (LLMs) into automated driving systems has opened new possibilities for reasoning and decision-making by transforming complex driving contexts into language-understandable representations. Recent studies demonstrate that fine-tuned LLMs can accurately predict …

• Steganographic Backdoor Attacks in NLP: Ultra-Low Poisoning and Defense Evasion

  Eric Xue, Ruiyi Zhang, Zijun Zhang, Pengtao Xie · 19. November 2025

  Transformer models are foundational to natural language processing (NLP) applications, yet remain vulnerable to backdoor attacks introduced through poisoned data, which implant hidden behaviors during training. To strengthen the ability to prevent such compromises, recent research has focused on des…

• DeepDefense: Layer-Wise Gradient-Feature Alignment for Building Robust Neural Networks

  Ci Lin, Tet Yeap, Iluju Kiringa, Biwei Zhang · 19. November 2025

  Deep neural networks are known to be vulnerable to adversarial perturbations, which are small and carefully crafted inputs that lead to incorrect predictions. In this paper, we propose DeepDefense, a novel defense framework that applies Gradient-Feature Alignment (GFA) regularization across multiple…

• Efficient Adversarial Malware Defense via Trust-Based Raw Override and Confidence-Adaptive Bit-Depth Reduction

  Ayush Chaudhary, Sisir Doppalpudi · 18. November 2025

  The deployment of robust malware detection systems in big data environments requires careful consideration of both security effectiveness and computational efficiency. While recent advances in adversarial defenses have demonstrated strong robustness improvements, they often introduce computational o…

• BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapse

  Yuanchao Wang, Tian Qin, Eduardo Valle, Bruno Abrahao · 18. November 2025

  Out-of-distribution (OOD) detection is critical for deploying image classifiers in safety-sensitive environments, yet existing detectors often struggle when OOD samples are semantically similar to the in-distribution (ID) classes. We present BootOOD, a fully self-supervised OOD detection framework t…

• Cost-Driven Synthesis of Sound Abstract Interpreters

  Qiuhan Gu, Avaljot Singh, Gagandeep Singh · 18. November 2025

  Constructing abstract interpreters that provide global soundness guarantees remains a major obstacle in abstract interpretation. We investigate whether modern LLMs can reduce this burden by leveraging them to synthesize sound, non-trivial abstract interpreters across multiple abstract domains in the…

• Exploiting Synergistic Cognitive Biases to Bypass Safety in LLMs

  Xikang Yang, Biyu Zhou, Xuehai Tang, Jizhong Han, Songlin Hu · 18. November 2025

  Large Language Models (LLMs) demonstrate impressive capabilities across a wide range of tasks, yet their safety mechanisms remain susceptible to adversarial attacks that exploit cognitive biases -- systematic deviations from rational judgment. Unlike prior jailbreaking approaches focused on prompt e…

• Nearest Neighbor Projection Removal Adversarial Training

  Himanshu Singh, A. V. Subramanyam, Shivank Rajput, Mohan Kankanhalli · 18. November 2025

  Deep neural networks have exhibited impressive performance in image classification tasks but remain vulnerable to adversarial examples. Standard adversarial training enhances robustness but typically fails to explicitly address inter-class feature overlap, a significant contributor to adversarial su…