Adversarial Robustness in Machine Learning

Neueste Paper

• Accelerating Targeted Hard-Label Adversarial Attacks in Low-Query Black-Box Settings

  Arjhun Swaminathan, Mete Akg\"un · 13. Januar 2026

  Deep neural networks for image classification remain vulnerable to adversarial examples -- small, imperceptible perturbations that induce misclassifications. In black-box settings, where only the final prediction is accessible, crafting targeted attacks that aim to misclassify into a specific target…

• A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model

  Qi Zheng, Shuliang Liu, Yu Huang, Sihang Jia, Jungang Li, Lyuhao Chen, Junhao Chen, Hanqian Li, Aiwei Liu, Yibo Yan, Xuming Hu · 13. Januar 2026

  Watermarking has emerged as a pivotal solution for content traceability and intellectual property protection in Large Vision-Language Models (LVLMs). However, vision-agnostic watermarks introduce visually irrelevant tokens and disrupt visual grounding by enforcing indiscriminate pseudo-random biases…

• When Should We Introduce Safety Interventions During Pretraining?

  Dylan Sam, Sachin Goyal, Pratyush Maini, Alexander Robey, J. Zico Kolter · 13. Januar 2026

  Ensuring the safety of language models in high-stakes settings remains a pressing challenge, as aligned behaviors are often brittle and easily undone by adversarial pressure or downstream finetuning. Prior work has shown that interventions applied during pretraining, such as rephrasing harmful conte…

• Reward-Preserving Attacks For Robust Reinforcement Learning

  Lucas Schott, Elies Gherbi, Hatem Hajri, Sylvain Lamprier · 13. Januar 2026

  Adversarial robustness in RL is difficult because perturbations affect entire trajectories: strong attacks can break learning, while weak attacks yield little robustness, and the appropriate strength varies by state. We propose $\alpha$-reward-preserving attacks, which adapt the strength of the adve…

• Hallucinations Live in Variance

  Aaron R. Flouro, Shawn P. Chadwick · 13. Januar 2026

  Benchmarks measure whether a model is correct. They do not measure whether a model is reliable. This distinction is largely academic for single-shot inference, but becomes critical for agentic AI systems, where a single rephrased prompt can trigger cascading failures in multi-step execution. Yet thi…

• Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment

  Haozhong Wang, Zhuo Li, Yibo Yang, He Zhao, Hongyuan Zha, Dandan Guo · 13. Januar 2026

  The inherent safety alignment of Large Language Models (LLMs) is prone to erosion during fine-tuning, even when using seemingly innocuous datasets. While existing defenses attempt to mitigate this via data selection, they typically rely on heuristic, instance-level assessments that neglect the globa…

• Defenses Against Prompt Attacks Learn Surface Heuristics

  Shawn Li, Chenxiao Yu, Zhiyu Ni, Hao Li, Charith Peris, Chaowei Xiao, Yue Zhao · 13. Januar 2026

  Large language models (LLMs) are increasingly deployed in security-sensitive applications, where they must follow system- or developer-specified instructions that define the intended task behavior, while completing benign user requests. When adversarial instructions appear in user queries or externa…

• Stress Testing Machine Learning at $10^{10}$ Scale: A Comprehensive Study of Adversarial Robustness on Algebraically Structured Integer Streams

  HyunJun Jeon · 13. Januar 2026

  This paper presents a large-scale stress test of machine learning systems using structured mathematical data as a benchmark. We evaluate the robustness of tree-based classifiers at an unprecedented scale, utilizing ten billion deterministic samples and five billion adversarial counterexamples. Our f…

• Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification

  Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng · 13. Januar 2026

  Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content, restricting their safe deployment. While traditional methods (e.g., alignment) adjust output preferences, they fail to eliminate underlying toxic regions in parameters, leaving models vu…

• SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization

  Xinghao Wang, Changtao Miao, Dianmo Sheng, Tao Gong, Qi Chu, Nenghai Yu, Quanchen Zou, Deyue Zhang, Xiangzheng Zhang · 13. Januar 2026

  Malicious image manipulation threatens public safety and requires efficient localization methods. Existing approaches depend on costly pixel-level annotations which make training expensive. Existing weakly supervised methods rely only on image-level binary labels and focus on global classification, …

• Towards Compositional Generalization in LLMs for Smart Contract Security: A Case Study on Reentrancy Vulnerabilities

  Ying Zhou, Jiacheng Wei, Yu Qi, Faguo Wu, Xiao Zhang · 13. Januar 2026

  Large language models (LLMs) demonstrate remarkable capabilities in natural language understanding and generation. Despite being trained on large-scale, high-quality data, LLMs still fail to outperform traditional static analysis tools in specialized domains like smart contract vulnerability detecti…

• Attack-Resistant Watermarking for AIGC Image Forensics via Diffusion-based Semantic Deflection

  Qingyu Liu, Yitao Zhang, Zhongjie Ba, Chao Shuai, Peng Cheng, Tianhang Zheng, Zhibo Wang · 13. Januar 2026

  Protecting the copyright of user-generated AI images is an emerging challenge as AIGC becomes pervasive in creative workflows. Existing watermarking methods (1) remain vulnerable to real-world adversarial threats, often forced to trade off between defenses against spoofing and removal attacks; and (…

• Safe Vision-Language Models via Unsafe Weights Manipulation

  Moreno D'Inc\`a, Elia Peruzzo, Xingqian Xu, Humphrey Shi, Nicu Sebe, Massimiliano Mancini · 13. Januar 2026

  Vision-language models (VLMs) often inherit the biases and unsafe associations present within their large-scale training dataset. While recent approaches mitigate unsafe behaviors, their evaluation focuses on how safe the model is on unsafe inputs, ignoring potential shortcomings on safe ones. In th…

• Zer0n: An AI-Assisted Vulnerability Discovery and Blockchain-Backed Integrity Framework

  Harshil Parmar, Pushti Vyas, Prayers Khristi, Priyank Panchal · 13. Januar 2026

  As vulnerability research increasingly adopts generative AI, a critical reliance on opaque model outputs has emerged, creating a "trust gap" in security automation. We address this by introducing Zer0n, a framework that anchors the reasoning capabilities of Large Language Models (LLMs) to the immuta…

• SafePro: Evaluating the Safety of Professional-Level AI Agents

  Kaiwen Zhou, Shreedhar Jangam, Ashwin Nagarajan, Tejas Polu, Suhas Oruganti, Chengzhi Liu, Ching-Chen Kuo, Yuting Zheng, Sravana Narayanaraju, Xin Eric Wang · 13. Januar 2026

  Large language model-based agents are rapidly evolving from simple conversational assistants into autonomous systems capable of performing complex, professional-level tasks in various domains. While these advancements promise significant productivity gains, they also introduce critical safety risks …

• Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs

  Weihao Hong, Zhiyuan Jiang, Bingyu Shen, Xinlei Guan, Yangyi Feng, Meng Xu, Boyang Li · 13. Januar 2026

  Vision-Language Models (VLMs) are increasingly used in safety-critical applications that require reliable visual grounding. However, these models often hallucinate details that are not present in the image to satisfy user prompts. While recent datasets and benchmarks have been introduced to evaluate…

• How to Backdoor the Knowledge Distillation

  Chen Wu, Qian Ma, Prasenjit Mitra, Sencun Zhu · 13. Januar 2026

  Knowledge distillation has become a cornerstone in modern machine learning systems, celebrated for its ability to transfer knowledge from a large, complex teacher model to a more efficient student model. Traditionally, this process is regarded as secure, assuming the teacher model is clean. This bel…

• AI Agent Smart Contract Exploit Generation

  Arthur Gervais, Liyi Zhou · 13. Januar 2026

  Smart contract vulnerabilities have led to billions in losses, yet finding actionable exploits remains challenging. Traditional fuzzers rely on rigid heuristics and struggle with complex attacks, while human auditors are thorough but slow and don't scale. Large Language Models offer a promising midd…

• Adversarial Attacks on Medical Hyperspectral Imaging Exploiting Spectral-Spatial Dependencies and Multiscale Features

  Yunrui Gu, Zhenzhe Gao, Cong Kong, Zhaoxia Yin · 13. Januar 2026

  Medical hyperspectral imaging (HSI) enables accurate disease diagnosis by capturing rich spectral-spatial tissue information, but recent advances in deep learning have exposed its vulnerability to adversarial attacks. In this work, we identify two fundamental causes of this fragility: the reliance o…

• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors

  Seongyun Lee, Yongrae Jo, Minju Seo, Moontae Lee, Minjoon Seo · 13. Januar 2026

  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information. However, this shift introduces input contexts that are inherently noisy, a reality that current sanitized benchmarks fail to capture. We introduce NoisyBench, a comprehensive…

• Parent-Guided Adaptive Reliability (PGAR): A Behavioural Meta-Learning Framework for Stable and Trustworthy AI

  Anshum Rankawat · 13. Januar 2026

  Parent-Guided Adaptive Reliability (PGAR) is a lightweight behavioural meta-learning framework that adds a supervisory "parent" layer on top of a standard learner to improve stability, calibration, and recovery under disturbances. PGAR computes three reflex-level signals (incident detection, overcon…

• Structure-Aware Diversity Pursuit as an AI Safety Strategy against Homogenization

  Ian Rios-Sialer · 13. Januar 2026

  Generative AI models reproduce the biases in the training data and can further amplify them through mode collapse. We refer to the resulting harmful loss of diversity as homogenization. Our position is that homogenization should be a primary concern in AI safety. We introduce xeno-reproduction as th…

• On Membership Inference Attacks in Knowledge Distillation

  Ziyao Cui, Minxing Zhang, Jian Pei · 13. Januar 2026

  Large language models (LLMs) are trained on massive corpora that may contain sensitive information, creating privacy risks under membership inference attacks (MIAs). Knowledge distillation is widely used to compress LLMs into smaller student models, but its privacy implications are poorly understood…

• AI Application Operations -- A Socio-Technical Framework for Data-driven Organizations

  Daniel J\"onsson, Mattias Tiger, Stefan Ekberg, Daniel Jakobsson, Mattias Jonhede, Fredrik Viksten · 13. Januar 2026

  We outline a comprehensive framework for artificial intelligence (AI) Application Operations (AIAppOps), based on real-world experiences from diverse organizations. Data-driven projects pose additional challenges to organizations due to their dependency on data across the development and operations …

• A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes

  Sahaya Jestus Lazer, Kshitiz Aryal, Maanak Gupta, Elisa Bertino · 12. Januar 2026

  Agentic AI marks an important transition from single-step generative models to systems capable of reasoning, planning, acting, and adapting over long-lasting tasks. By integrating memory, tool use, and iterative decision cycles, these systems enable continuous, autonomous workflows in real-world env…