Adversarial Robustness in Machine Learning

Neueste Paper

• How to Trick Your AI TA: A Systematic Study of Academic Jailbreaking in LLM Code Evaluation

  Devanshu Sahoo, Vasudev Majhi, Arjun Neekhra, Yash Sinha, Murari Mandal, Dhruv Kumar · 12. Dezember 2025

  The use of Large Language Models (LLMs) as automatic judges for code evaluation is becoming increasingly prevalent in academic environments. But their reliability can be compromised by students who may employ adversarial prompting strategies in order to induce misgrading and secure undeserved academ…

• UACER: An Uncertainty-Aware Critic Ensemble Framework for Robust Adversarial Reinforcement Learning

  Jiaxi Wu, Tiantian Zhang, Yuxing Wang, Yongzhe Chang, Xueqian Wang · 12. Dezember 2025

  Robust adversarial reinforcement learning has emerged as an effective paradigm for training agents to handle uncertain disturbance in real environments, with critical applications in sequential decision-making domains such as autonomous driving and robotic control. Within this paradigm, agent traini…

• PaTAS: A Framework for Trust Propagation in Neural Networks Using Subjective Logic

  Koffi Ismael Ouattara, Ioannis Krontiris, Theo Dimitrakos, Dennis Eisermann, Houda Labiod, Frank Kargl · 12. Dezember 2025

  Trustworthiness has become a key requirement for the deployment of artificial intelligence systems in safety-critical applications. Conventional evaluation metrics, such as accuracy and precision, fail to appropriately capture uncertainty or the reliability of model predictions, particularly under a…

• SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination

  Sangha Park, Seungryong Yoo, Jisoo Mok, Sungroh Yoon · 12. Dezember 2025

  Although Multimodal Large Language Models (MLLMs) have advanced substantially, they remain vulnerable to object hallucination caused by language priors and visual information loss. To address this, we propose SAVE (Sparse Autoencoder-Driven Visual Information Enhancement), a framework that mitigates…

• Authority Backdoor: A Certifiable Backdoor Mechanism for Authoring DNNs

  Han Yang, Shaofeng Li, Tian Dong, Xiangyu Xu, Guangchi Liu, Zhen Ling · 12. Dezember 2025

  Deep Neural Networks (DNNs), as valuable intellectual property, face unauthorized use. Existing protections, such as digital watermarking, are largely passive; they provide only post-hoc ownership verification and cannot actively prevent the illicit use of a stolen model. This work proposes a proact…

• Empirical evaluation of the Frank-Wolfe methods for constructing white-box adversarial attacks

  Kristina Korotkova, Aleksandr Katrutsa · 12. Dezember 2025

  The construction of adversarial attacks for neural networks appears to be a crucial challenge for their deployment in various services. To estimate the adversarial robustness of a neural network, a fast and efficient approach is needed to construct adversarial attacks. Since the formalization of adv…

• Uncertainty-Preserving QBNNs: Multi-Level Quantization of SVI-Based Bayesian Neural Networks for Image Classification

  Hendrik Borras, Yong Wu, Bernhard Klein, Holger Fr\"oning · 12. Dezember 2025

  Bayesian Neural Networks (BNNs) provide principled uncertainty quantification but suffer from substantial computational and memory overhead compared to deterministic networks. While quantization techniques have successfully reduced resource requirements in standard deep learning models, their applic…

• Verifying LLM Inference to Detect Model Weight Exfiltration

  Roy Rinberg, Adam Karvonen, Alexander Hoover, Daniel Reuter, Keri Warr · 12. Dezember 2025

  As large AI models become increasingly valuable assets, the risk of model weight exfiltration from inference servers grows accordingly. An attacker controlling an inference server may exfiltrate model weights by hiding them within ordinary model outputs, a strategy known as steganography. This work …

• RobustSora: De-Watermarked Benchmark for Robust AI-Generated Video Detection

  Zhuo Wang, Xiliang Liu, Ligang Sun · 12. Dezember 2025

  The proliferation of AI-generated video technologies poses challenges to information integrity. While recent benchmarks advance AIGC video detection, they overlook a critical factor: many state-of-the-art generative models embed digital watermarks in outputs, and detectors may partially rely on thes…

• The Eminence in Shadow: Exploiting Feature Boundary Ambiguity for Robust Backdoor Attacks

  Zhou Feng, Jiahao Chen, Chunyi Zhou, Yuwen Pu, Tianyu Du, Jinbao Li, Jianhai Chen, Shouling Ji · 12. Dezember 2025

  Deep neural networks (DNNs) underpin critical applications yet remain vulnerable to backdoor attacks, typically reliant on heuristic brute-force methods. Despite significant empirical advancements in backdoor research, the lack of rigorous theoretical analysis limits understanding of underlying mech…

• From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection

  Chaomeng Lu, Bert Lagaisse · 12. Dezember 2025

  Vulnerability detection methods based on deep learning (DL) have shown strong performance on benchmark datasets, yet their real-world effectiveness remains underexplored. Recent work suggests that both graph neural network (GNN)-based and transformer-based models, including large language models (LL…

• ZK-APEX: Zero-Knowledge Approximate Personalized Unlearning with Executable Proofs

  Mohammad M Maheri, Sunil Cotterill, Alex Davidson, Hamed Haddadi · 12. Dezember 2025

  Machine unlearning aims to remove the influence of specific data points from a trained model to satisfy privacy, copyright, and safety requirements. In real deployments, providers distribute a global model to many edge devices, where each client personalizes the model using private data. When a dele…

• V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions

  Mumuksh Tayal, Manan Tayal, Aditya Singh, Shishir Kolathaya, Ravi Prakash · 12. Dezember 2025

  Ensuring safety in autonomous systems requires controllers that satisfy hard, state-wise constraints without relying on online interaction. While existing Safe Offline RL methods typically enforce soft expected-cost constraints, they do not guarantee forward invariance. Conversely, Control Barrier F…

• Boosting RL-Based Visual Reasoning with Selective Adversarial Entropy Intervention

  Yang Yu, Zhuangzhuang Chen, Siqi Wang, Lanqing Li, Xiaomeng Li · 12. Dezember 2025

  Recently, reinforcement learning (RL) has become a common choice in enhancing the reasoning capabilities of vision-language models (VLMs). Considering existing RL- based finetuning methods, entropy intervention turns out to be an effective way to benefit exploratory ability, thereby improving policy…

• Trustworthy Orchestration Artificial Intelligence by the Ten Criteria with Control-Plane Governance

  Byeong Ho Kang, Wenli Yang, Muhammad Bilal Amin · 12. Dezember 2025

  As Artificial Intelligence (AI) systems increasingly assume consequential decision-making roles, a widening gap has emerged between technical capabilities and institutional accountability. Ethical guidance alone is insufficient to counter this challenge; it demands architectures that embed governanc…

• Metaphor-based Jailbreaking Attacks on Text-to-Image Models

  Chenyu Zhang, Yiwen Ma, Lanjun Wang, Wenhui Li, Yi Tu, An-An Liu · 12. Dezember 2025

  Text-to-image~(T2I) models commonly incorporate defense mechanisms to prevent the generation of sensitive images. Unfortunately, recent jailbreaking attacks have shown that adversarial prompts can effectively bypass these mechanisms and induce T2I models to produce sensitive content, revealing criti…

• Natural Geometry of Robust Data Attribution: From Convex Models to Deep Networks

  Shihao Li, Jiachen Li, Dongmei Chen · 11. Dezember 2025

  Data attribution methods identify which training examples are responsible for a model's predictions, but their sensitivity to distributional perturbations undermines practical reliability. We present a unified framework for certified robust attribution that extends from convex models to deep network…

• CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning

  Mingyuan Li, Chunyu Liu, Zhuojun Li, Xiao Liu, Guangsheng Yu, Bo Du, Jun Shen, Qiang Wu · 11. Dezember 2025

  Traffic accidents result in millions of injuries and fatalities globally, with a significant number occurring at intersections each year. Traffic Signal Control (TSC) is an effective strategy for enhancing safety at these urban junctures. Despite the growing popularity of Reinforcement Learning (RL)…

• Deterministic World Models for Verification of Closed-loop Vision-based Systems

  Yuang Geng, Zhuoyang Zhou, Zhongzheng Zhang, Siyuan Pan, Hoang-Dung Tran, Ivan Ruchkin · 11. Dezember 2025

  Verifying closed-loop vision-based control systems remains a fundamental challenge due to the high dimensionality of images and the difficulty of modeling visual environments. While generative models are increasingly used as camera surrogates in verification, their reliance on stochastic latent vari…

• Robust and Sparse Estimation of Unbounded Density Ratio under Heavy Contamination

  Ryosuke Nagumo, Hironori Fujisawa · 11. Dezember 2025

  We examine the non-asymptotic properties of robust density ratio estimation (DRE) in contaminated settings. Weighted DRE is the most promising among existing methods, exhibiting doubly strong robustness from an asymptotic perspective. This study demonstrates that Weighted DRE achieves sparse consist…

• Impact of Positional Encoding: Clean and Adversarial Rademacher Complexity for Transformers under In-Context Regression

  Weiyi He, Yue Xing · 11. Dezember 2025

  Positional encoding (PE) is a core architectural component of Transformers, yet its impact on the Transformer's generalization and robustness remains unclear. In this work, we provide the first generalization analysis for a single-layer Transformer under in-context regression that explicitly account…

• Black-Box Behavioral Distillation Breaks Safety Alignment in Medical LLMs

  Sohely Jahan, Ruimin Sun · 11. Dezember 2025

  As medical large language models (LLMs) become increasingly integrated into clinical workflows, concerns around alignment robustness, and safety are escalating. Prior work on model extraction has focused on classification models or memorization leakage, leaving the vulnerability of safety-aligned ge…

• Hard Work Does Not Always Pay Off: Poisoning Attacks on Neural Architecture Search

  Zachary Coalson, Huazheng Wang, Qingyun Wu, Sanghyun Hong · 11. Dezember 2025

  We study the robustness of data-centric methods to find neural network architectures, known as neural architecture search (NAS), against data poisoning. To audit this robustness, we design a poisoning framework that enables the systematic evaluation of the ability of NAS to produce architectures und…

• Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners

  Soichiro Kumano, Hiroshi Kera, Toshihiko Yamasaki · 11. Dezember 2025

  Adversarial training is one of the most effective adversarial defenses, but it incurs a high computational cost. In this study, we present the first theoretical analysis suggesting that adversarially pretrained transformers can serve as universally robust foundation models -- models that can robustl…

• Colliding with Adversaries at ECML-PKDD 2025 Model Robustness Competition 1st Prize Solution

  Dimitris Stefanopoulos, Andreas Voskou · 11. Dezember 2025

  This report presents the winning solution for Task 2 of Colliding with Adversaries: A Challenge on Robust Learning in High Energy Physics Discovery at ECML-PKDD 2025. The goal of the challenge was to design and train a robust ANN-based model capable of achieving high accuracy in a binary classificat…