Adversarial Robustness in Machine Learning

Últimos artículos

• GradID: Adversarial Detection via Intrinsic Dimensionality of Gradients

  Mohammad Mahdi Razmjoo, Mohammad Mahdi Sharifian, Saeed Bagheri Shouraki · 16 de diciembre de 2025

  Despite their remarkable performance, deep neural networks exhibit a critical vulnerability: small, often imperceptible, adversarial perturbations can lead to drastically altered model predictions. Given the stringent reliability demands of applications such as medical diagnosis and autonomous drivi…

• A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks

  S M Asif Hossain, Ruksat Khan Shayoni, Mohd Ruhul Ameen, Akif Islam, M. F. Mridha, Jungpil Shin · 16 de diciembre de 2025

  Prompt injection attacks represent a major vulnerability in Large Language Model (LLM) deployments, where malicious instructions embedded in user inputs can override system prompts and induce unintended behaviors. This paper presents a novel multi-agent defense framework that employs specialized LLM…

• BreakFun: Jailbreaking LLMs via Schema Exploitation

  Amirkia Rafiei Oskooei, Mehmet S. Aktas · 16 de diciembre de 2025

  The proficiency of Large Language Models (LLMs) in processing structured data and adhering to syntactic rules is a capability that drives their widespread adoption but also makes them paradoxically vulnerable. In this paper, we investigate this vulnerability through BreakFun, a jailbreak methodology…

• Uncertainty Quantification for Machine Learning: One Size Does Not Fit All

  Paul Hofman, Yusuf Sale, Eyke H\"ullermeier · 16 de diciembre de 2025

  Proper quantification of predictive uncertainty is essential for the use of machine learning in safety-critical applications. Various uncertainty measures have been proposed for this purpose, typically claiming superiority over other measures. In this paper, we argue that there is no single best mea…

• Defending Collaborative Filtering Recommenders via Adversarial Robustness Based Edge Reweighting

  Yongyu Wang · 16 de diciembre de 2025

  User based collaborative filtering (CF) relies on a user and user similarity graph, making it vulnerable to profile injection (shilling) attacks that manipulate neighborhood relations to promote (push) or demote (nuke) target items. In this work, we propose an adversarial robustness based edge rewei…

• Behavior-Aware and Generalizable Defense Against Black-Box Adversarial Attacks for ML-Based IDS

  Sabrine Ennaji, Elhadj Benkhelifa, Luigi Vincenzo Mancini · 16 de diciembre de 2025

  Machine learning based intrusion detection systems are increasingly targeted by black box adversarial attacks, where attackers craft evasive inputs using indirect feedback such as binary outputs or behavioral signals like response time and resource usage. While several defenses have been proposed, i…

• Selective Conformal Risk Control

  Yunpeng Xu, Wenge Guo, Zhi Wei · 16 de diciembre de 2025

  Reliable uncertainty quantification is essential for deploying machine learning systems in high-stakes domains. Conformal prediction provides distribution-free coverage guarantees but often produces overly large prediction sets, limiting its practical utility. To address this issue, we propose \text…

• ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts

  Darpan Aswal, C\'eline Hudelot · 16 de diciembre de 2025

  Large Language Models have found success in a variety of applications. However, their safety remains a concern due to the existence of various jailbreaking methods. Despite significant efforts, alignment and safety fine-tuning only provide a certain degree of robustness against jailbreak attacks tha…

• Learning to Extract Context for Context-Aware LLM Inference

  Minseon Kim, Lucas Caccia, Zhengyan Shi, Matheus Pereira, Marc-Alexandre C\^ot\'e, Xingdi Yuan, Alessandro Sordoni · 16 de diciembre de 2025

  User prompts to large language models (LLMs) are often ambiguous or under-specified, and subtle contextual cues shaped by user intentions, prior knowledge, and risk factors strongly influence what constitutes an appropriate response. Misinterpreting intent or risks may lead to unsafe outputs, while …

• CLOAK: Contrastive Guidance for Latent Diffusion-Based Data Obfuscation

  Xin Yang, Omid Ardakanian · 16 de diciembre de 2025

  Data obfuscation is a promising technique for mitigating attribute inference attacks by semi-trusted parties with access to time-series data emitted by sensors. Recent advances leverage conditional generative models together with adversarial training or mutual information-based regularization to bal…

• Keep the Lights On, Keep the Lengths in Check: Plug-In Adversarial Detection for Time-Series LLMs in Energy Forecasting

  Hua Ma, Ruoxi Sun, Minhui Xue, Xingliang Yuan, Carsten Rudolph, Surya Nepal, Ling Liu · 16 de diciembre de 2025

  Accurate time-series forecasting is increasingly critical for planning and operations in low-carbon power systems. Emerging time-series large language models (TS-LLMs) now deliver this capability at scale, requiring no task-specific retraining, and are quickly becoming essential components within th…

• Calibrating Uncertainty for Zero-Shot Adversarial CLIP

  Wenjing lu, Zerui Tao, Dongping Zhang, Yuning Qiu, Yang Yang, Qibin Zhao · 16 de diciembre de 2025

  CLIP delivers strong zero-shot classification but remains highly vulnerable to adversarial attacks. Previous work of adversarial fine-tuning largely focuses on matching the predicted logits between clean and adversarial examples, which overlooks uncertainty calibration and may degrade the zero-shot …

• Backdoors in DRL: Four Environments Focusing on In-distribution Triggers

  Chace Ashcraft, Ted Staley, Josh Carney, Cameron Hickert, Derek Juba, Kiran Karra, Nathan Drenkow · 16 de diciembre de 2025

  Backdoor attacks, or trojans, pose a security risk by concealing undesirable behavior in deep neural network models. Open-source neural networks are downloaded from the internet daily, possibly containing backdoors, and third-party model developers are common. To advance research on backdoor attack …

• Quantifying Robustness: A Benchmarking Framework for Deep Learning Forecasting in Cyber-Physical Systems

  Alexander Windmann, Henrik Steude, Daniel Boschmann, Oliver Niggemann · 16 de diciembre de 2025

  Cyber-Physical Systems (CPS) in domains such as manufacturing and energy distribution generate complex time series data crucial for Prognostics and Health Management (PHM). While Deep Learning (DL) methods have demonstrated strong forecasting capabilities, their adoption in industrial CPS remains li…

• Trojan Cleansing with Neural Collapse

  Xihe Gu, Greg Fields, Yaman Jandali, Tara Javidi, Farinaz Koushanfar · 16 de diciembre de 2025

  Trojan attacks are sophisticated training-time attacks on neural networks that embed backdoor triggers which force the network to produce a specific output on any input which includes the trigger. With the increasing relevance of deep networks which are too large to train with personal resources and…

• Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring

  Peichun Hua, Hao Li, Shanghao Shi, Zhiyuan Yu, Ning Zhang · 16 de diciembre de 2025

  Large Vision-Language Models (LVLMs) are vulnerable to a growing array of multimodal jailbreak attacks, necessitating defenses that are both generalizable to novel threats and efficient for practical deployment. Many current strategies fall short, either targeting specific attack patterns, which lim…

• DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective

  Shuo Shao, Yiming Li, Mengren Zheng, Zhiyang Hu, Yukun Chen, Boheng Li, Yu He, Junfeng Guo, Dacheng Tao, Zhan Qin · 16 de diciembre de 2025

  The widespread application of Deep Learning across diverse domains hinges critically on the quality and composition of training datasets. However, the common lack of disclosure regarding their usage raises significant privacy and copyright concerns. Dataset auditing techniques, which aim to determin…

• Cisco Integrated AI Security and Safety Framework Report

  Amy Chang, Tiffany Saade, Sanket Mendapara, Adam Swanda, Ankit Garg · 16 de diciembre de 2025

  Artificial intelligence (AI) systems are being readily and rapidly adopted, increasingly permeating critical domains: from consumer platforms and enterprise software to networked systems with embedded agents. While this has unlocked potential for human productivity gains, the attack surface has expa…

• Explainable Adversarial-Robust Vision-Language-Action Model for Robotic Manipulation

  Ju-Young Kim, Ji-Hong Park, Myeongjun Kim, Gun-Woo Kim · 16 de diciembre de 2025

  Smart farming has emerged as a key technology for advancing modern agriculture through automation and intelligent control. However, systems relying on RGB cameras for perception and robotic manipulators for control, common in smart farming, are vulnerable to photometric perturbations such as hue, il…

• Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors

  Max McGuinness, Alex Serrano, Luke Bailey, Scott Emmons · 16 de diciembre de 2025

  Activation monitoring, which probes a model's internal states using lightweight classifiers, is an emerging tool for AI safety. However, its worst-case robustness under a misalignment threat model--where a model might learn to actively conceal its internal states--remains untested. Focusing on this …

• A Technical Policy Blueprint for Trustworthy Decentralized AI

  Hasan Kassem, Sergen Cansiz, Brandon Edwards, Patrick Foley, Inken Hagestedt, Taeho Jung, Prakash Moorthy, Michael O'Connor, Bruno Rodrigues, Holger Roth, Micah Sheller, Dimitris Stripelis, Marc Vesin, Renato Umeton, Mic Bowman, Alexandros Karargyris · 16 de diciembre de 2025

  Decentralized AI systems, such as federated learning, can play a critical role in further unlocking AI asset marketplaces (e.g., healthcare data marketplaces) thanks to increased asset privacy protection. Unlocking this big potential necessitates governance mechanisms that are transparent, scalable,…

• Robustness of Probabilistic Models to Low-Quality Data: A Multi-Perspective Analysis

  Liu Peng, Yaochu Jin · 16 de diciembre de 2025

  A systematic, comparative investigation into the effects of low-quality data reveals a stark spectrum of robustness across modern probabilistic models. We find that autoregressive language models, from token prediction to sequence-to-sequence tasks, are remarkably resilient (for GPT-2, test NLL incr…

• From Overfitting to Reliability: Introducing the Hierarchical Approximate Bayesian Neural Network

  Hayk Amirkhanian, Marco F. Huber · 16 de diciembre de 2025

  In recent years, neural networks have revolutionized various domains, yet challenges such as hyperparameter tuning and overfitting remain significant hurdles. Bayesian neural networks offer a framework to address these challenges by incorporating uncertainty directly into the model, yielding more re…

• Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability

  Leonard Bereska, Zoe Tzifa-Kratira, Reza Samavi, Efstratios Gavves · 16 de diciembre de 2025

  Neural networks achieve remarkable performance through superposition: encoding multiple features as overlapping directions in activation space rather than dedicating individual neurons to each feature. This challenges interpretability, yet we lack principled methods to measure superposition. We pres…

• Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification

  Duo Zhou, Jorge Chavez, Hesun Chen, Grani A. Hanasusanto, Huan Zhang · 15 de diciembre de 2025

  State-of-the-art neural network (NN) verifiers demonstrate that applying the branch-and-bound (BaB) procedure with fast bounding techniques plays a key role in tackling many challenging verification properties. In this work, we introduce the linear constraint-driven clipping framework, a class of sc…