Adversarial Robustness in Machine Learning

Neueste Paper

• CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications

  Raviraj Joshi, Rakesh Paul, Kanishk Singla, Anusha Kamath, Michael Evans, Katherine Luna, Shaona Ghosh, Utkarsh Vaidya, Eileen Long, Sanjay Singh Chauhan, Niranjan Wartikar · 11. November 2025

  The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We p…

• GPT, But Backwards: Exactly Inverting Language Model Outputs

  Adrians Skapars, Edoardo Manino, Youcheng Sun, Lucas C. Cordeiro · 11. November 2025

  The task of reconstructing unknown textual inputs to language models is a fundamental auditing primitive that allows us to assess the model's vulnerability to a range of security issues, including stealing hidden system prompts, detecting backdoors, and leaking private data. Existing inversion works…

• Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach

  Junchao Fan, Qi Wei, Ruichen Zhang, Dusit Niyato, Yang Lu, Jianhua Wang, Xiaolin Chang, Bo Ai · 11. November 2025

  Deep reinforcement learning (DRL) has demonstrated remarkable success in developing autonomous driving policies. However, its vulnerability to adversarial attacks remains a critical barrier to real-world deployment. Although existing robust methods have achieved success, they still suffer from three…

• DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning

  Yaxuan Wang, Chris Yuhao Liu, Quan Liu, Jinglong Pang, Wei Wei, Yujia Bao, Yang Liu · 11. November 2025

  Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain da…

• Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey

  Albert Schotschneider, Svetlana Pavlitska, J. Marius Z\"ollner · 11. November 2025

  Deep neural networks (DNNs) are widely used in perception systems for safety-critical applications, such as autonomous driving and robotics. However, DNNs remain vulnerable to various safety concerns, including generalization errors, out-of-distribution (OOD) inputs, and adversarial attacks, which c…

• Federated Learning for Video Violence Detection: Complementary Roles of Lightweight CNNs and Vision-Language Models for Energy-Efficient Use

  S\'ebastien Thuau, Siba Haidar, Rachid Chelouah · 11. November 2025

  Deep learning-based video surveillance increasingly demands privacy-preserving architectures with low computational and environmental overhead. Federated learning preserves privacy but deploying large vision-language models (VLMs) introduces major energy and sustainability challenges. We compare thr…

• Differentiated Directional Intervention A Framework for Evading LLM Safety Alignment

  Peng Zhang, peijie sun · 11. November 2025

  Safety alignment instills in Large Language Models (LLMs) a critical capacity to refuse malicious requests. Prior works have modeled this refusal mechanism as a single linear direction in the activation space. We posit that this is an oversimplification that conflates two functionally distinct neura…

• EASE: Practical and Efficient Safety Alignment for Small Language Models

  Haonan Shi, Guoli Wang, Tu Ouyang, An Wang · 11. November 2025

  Small language models (SLMs) are increasingly deployed on edge devices, making their safety alignment crucial yet challenging. Current shallow alignment methods that rely on direct refusal of malicious queries fail to provide robust protection, particularly against adversarial jailbreaks. While deli…

• Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Models

  Jaesin Ahn, Heechul Jung · 11. November 2025

  Diffusion models show remarkable image generation performance following text prompts, but risk generating sexual contents. Existing approaches, such as prompt filtering, concept removal, and even sexual contents mitigation methods, struggle to defend against adversarial attacks while maintaining ben…

• Breaking the Stealth-Potency Trade-off in Clean-Image Backdoors with Generative Trigger Optimization

  Binyan Xu, Fan Yang, Di Tang, Xilin Dai, Kehuan Zhang · 11. November 2025

  Clean-image backdoor attacks, which use only label manipulation in training datasets to compromise deep neural networks, pose a significant threat to security-critical applications. A critical flaw in existing methods is that the poison rate required for a successful attack induces a proportional, a…

• Uniform Convergence of Adversarially Robust Classifiers

  Rachel Morris, Ryan Murray · 11. November 2025

  In recent years there has been significant interest in the effect of different types of adversarial perturbations in data classification problems. Many of these models incorporate the adversarial power, which is an important parameter with an associated trade-off between accuracy and robustness. Thi…

• VMDT: Decoding the Trustworthiness of Video Foundation Models

  Yujin Potter, Zhun Wang, Nicholas Crispino, Kyle Montgomery, Alexander Xiong, Ethan Y. Chang, Francesco Pinto, Yuqi Chen, Rahul Gupta, Morteza Ziyadi, Christos Christodoulopoulos, Bo Li, Chenguang Wang, Dawn Song · 11. November 2025

  As foundation models become more sophisticated, ensuring their trustworthiness becomes increasingly critical; yet, unlike text and image, the video modality still lacks comprehensive trustworthiness benchmarks. We introduce VMDT (Video-Modal DecodingTrust), the first unified platform for evaluating …

• SMiLE: Provably Enforcing Global Relational Properties in Neural Networks

  Matteo Francobaldi, Michele Lombardi, Andrea Lodi · 11. November 2025

  Artificial Intelligence systems are increasingly deployed in settings where ensuring robustness, fairness, or domain-specific properties is essential for regulation compliance and alignment with human values. However, especially on Neural Networks, property enforcement is very challenging, and exist…

• Probably Approximately Global Robustness Certification

  Peter Blohm, Patrick Indri, Thomas G\"artner, Sagar Malhotra · 11. November 2025

  We propose and investigate probabilistic guarantees for the adversarial robustness of classification algorithms. While traditional formal verification approaches for robustness are intractable and sampling-based approaches do not provide formal guarantees, our approach is able to efficiently certify…

• Scalable Verification of Neural Control Barrier Functions Using Linear Bound Propagation

  Nikolaus Vertovec, Frederik Baymler Mathiesen, Thom Badings, Luca Laurenti, Alessandro Abate · 11. November 2025

  Control barrier functions (CBFs) are a popular tool for safety certification of nonlinear dynamical control systems. Recently, CBFs represented as neural networks have shown great promise due to their expressiveness and applicability to a broad class of dynamics and safety constraints. However, veri…

• CatBack: Universal Backdoor Attacks on Tabular Data via Categorical Encoding

  Behrad Tajalli, Stefanos Koffas, Stjepan Picek · 11. November 2025

  Backdoor attacks in machine learning have drawn significant attention for their potential to compromise models stealthily, yet most research has focused on homogeneous data such as images. In this work, we propose a novel backdoor attack on tabular data, which is particularly challenging due to the …

• Catching Contamination Before Generation: Spectral Kill Switches for Agents

  Valentin No\"el · 11. November 2025

  Agentic language models compose multi step reasoning chains, yet intermediate steps can be corrupted by inconsistent context, retrieval errors, or adversarial inputs, which makes post hoc evaluation too late because errors propagate before detection. We introduce a diagnostic that requires no additi…

• TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification

  Pasan Dissanayake, Sanghamitra Dutta · 11. November 2025

  Transformer-based models have shown promising performance on tabular data compared to their classical counterparts such as neural networks and Gradient Boosted Decision Trees (GBDTs) in scenarios with limited training data. They utilize their pre-trained knowledge to adapt to new domains, achieving …

• Enhancing Adversarial Robustness of IoT Intrusion Detection via SHAP-Based Attribution Fingerprinting

  Dilli Prasad Sharma, Liang Xue, Xiaowei Sun, Xiaodong Lin, Pulei Xiong · 11. November 2025

  The rapid proliferation of Internet of Things (IoT) devices has transformed numerous industries by enabling seamless connectivity and data-driven automation. However, this expansion has also exposed IoT networks to increasingly sophisticated security threats, including adversarial attacks targeting …

• Jailbreaking in the Haystack

  Rishi Rajesh Shah, Chen Henry Wu, Shashwat Saxena, Ziqian Zhong, Alexander Robey, Aditi Raghunathan · 10. November 2025

  Recent advances in long-context language models (LMs) have enabled million-token inputs, expanding their capabilities across complex tasks like computer-use agents. Yet, the safety implications of these extended contexts remain unclear. To bridge this gap, we introduce NINJA (short for Needle-in-hay…

• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking

  Wenyuan Yang, Yichen Sun, Changzheng Chen, Zhixuan Chu, Jiaheng Zhang, Yiming Li, Dacheng Tao · 10. November 2025

  Large-scale vision-language models, especially CLIP, have demonstrated remarkable performance across diverse downstream tasks. Soft prompts, as carefully crafted modules that efficiently adapt vision-language models to specific tasks, necessitate effective copyright protection. In this paper, we inv…

• Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies

  Prasoon Varshney, Makesh Narsimhan Sreedhar, Liwei Jiang, Traian Rebedea, Christopher Parisien · 10. November 2025

  Large language models (LLMs) are typically aligned to a universal set of safety and usage principles intended for broad public acceptability. Yet, real-world applications of LLMs often take place within organizational ecosystems shaped by distinctive corporate policies, regulatory requirements, use …

• A Metamorphic Testing Perspective on Knowledge Distillation for Language Models of Code: Does the Student Deeply Mimic the Teacher?

  Md. Abdul Awal, Mrigank Rochan, Chanchal K. Roy · 10. November 2025

  Transformer-based language models of code have achieved state-of-the-art performance across a wide range of software analytics tasks, but their practical deployment remains limited due to high computational costs, slow inference speeds, and significant environmental impact. To address these challeng…

• Conformal Prediction Adaptive to Unknown Subpopulation Shifts

  Nien-Shao Wang, Duygu Nur Yaldiz, Yavuz Faruk Bakman, Sai Praneeth Karimireddy · 10. November 2025

  Conformal prediction is widely used to equip black-box machine learning models with uncertainty quantification, offering formal coverage guarantees under exchangeable data. However, these guarantees fail when faced with subpopulation shifts, where the test environment contains a different mix of sub…

• A Perfectly Truthful Calibration Measure

  Jason Hartline, Lunjia Hu, Yifan Wu · 10. November 2025

  Calibration requires that predictions are conditionally unbiased and, therefore, reliably interpretable as probabilities. A calibration measure quantifies how far a predictor is from perfect calibration. As introduced by Haghtalab et al. (2024), a calibration measure is truthful if it is minimized i…