Adversarial Robustness in Machine Learning

Últimos artículos

• Visual Memory Injection Attacks for Multi-Turn Conversations

  Christian Schlarmann, Matthias Hein · 19 de febrero de 2026

  Generative large vision-language models (LVLMs) have recently achieved impressive performance gains, and their user base is growing rapidly. However, the security of LVLMs, in particular in a long-context multi-turn setting, is largely underexplored. In this paper, we consider the realistic scenario…

• Protecting the Undeleted in Machine Unlearning

  Aloni Cohen, Refael Kohen, Kobbi Nissim, Uri Stemmer · 19 de febrero de 2026

  Machine unlearning aims to remove specific data points from a trained model, often striving to emulate "perfect retraining", i.e., producing the model that would have been obtained had the deleted data never been included. We demonstrate that this approach, and security definitions that enable it, c…

• Recursive language models for jailbreak detection: a procedural defense for tool-augmented agents

  Doron Shavit · 19 de febrero de 2026

  Jailbreak prompts are a practical and evolving threat to large language models (LLMs), particularly in agentic systems that execute tools over untrusted content. Many attacks exploit long-context hiding, semantic camouflage, and lightweight obfuscations that can evade single-pass guardrails. We pres…

• Vulnerability Analysis of Safe Reinforcement Learning via Inverse Constrained Reinforcement Learning

  Jialiang Fan, Shixiong Jiang, Mengyu Liu, Fanxin Kong · 19 de febrero de 2026

  Safe reinforcement learning (Safe RL) aims to ensure policy performance while satisfying safety constraints. However, most existing Safe RL methods assume benign environments, making them vulnerable to adversarial perturbations commonly encountered in real-world settings. In addition, existing gradi…

• Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation

  Daniel Bethell, Simos Gerasimou, Radu Calinescu, Calum Imrie · 19 de febrero de 2026

  Ensuring the safe exploration of reinforcement learning (RL) agents is critical for deployment in real-world systems. Yet existing approaches struggle to strike the right balance: methods that tightly enforce safety often cripple task performance, while those that prioritize reward leave safety cons…

• Navigating the Deep: End-to-End Extraction on Deep Neural Networks

  Haolin Liu, Adrien Siproudhis, Samuel Experton, Peter Lorenz, Christina Boura, Thomas Peyrin · 19 de febrero de 2026

  Neural network model extraction has recently emerged as an important security concern, as adversaries attempt to recover a network's parameters via black-box queries. Carlini et al. proposed in CRYPTO'20 a model extraction approach, consisting of two steps: signature extraction and sign extraction. …

• A Note on Non-Composability of Layerwise Approximate Verification for Neural Inference

  Or Zamir · 18 de febrero de 2026

  A natural and informal approach to verifiable (or zero-knowledge) ML inference over floating-point data is: ``prove that each layer was computed correctly up to tolerance $\delta$; therefore the final output is a reasonable inference result''. This short note gives a simple counterexample showing th…

• MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference

  Rapha\"el Baur, Yannick Metz, Maria Gkoulta, Mennatallah El-Assady, Giorgia Ramponi, Thomas Kleine Buening · 18 de febrero de 2026

  Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it remains unclear how to jointly learn reward functions from heterogeneous feedback types such as demonstrations, comparisons, ratings, and stops that provid…

• Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation

  Shojiro Yamabe, Kazuto Fukuchi, Jun Sakuma · 18 de febrero de 2026

  This study investigates behavior-targeted attacks on reinforcement learning and their countermeasures. Behavior-targeted attacks aim to manipulate the victim's behavior as desired by the adversary through adversarial interventions in state observations. Existing behavior-targeted attacks have some l…

• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety

  Sanidhya Vijayvargiya, Aditya Bharat Soni, Xuhui Zhou, Zora Zhiruo Wang, Nouha Dziri, Graham Neubig, Maarten Sap · 18 de febrero de 2026

  Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilities for unsafe behavior demands rigorous evaluation. While prior benchmarks have attempted to assess agent safety, most fa…

• The Stationarity Bias: Stratified Stress-Testing for Time-Series Imputation in Regulated Dynamical Systems

  Amirreza Dolatpour Fathkouhi, Alireza Namazi, Heman Shakeri · 18 de febrero de 2026

  Time-series imputation benchmarks employ uniform random masking and shape-agnostic metrics (MSE, RMSE), implicitly weighting evaluation by regime prevalence. In systems with a dominant attractor -- homeostatic physiology, nominal industrial operation, stable network traffic -- this creates a systema…

• RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses

  Mohamed Djilani, Salah Ghamizi, Maxime Cordy · 18 de febrero de 2026

  Although adversarial robustness has been extensively studied in white-box settings, recent advances in black-box attacks (including transfer- and query-based approaches) are primarily benchmarked against weak defenses, leaving a significant gap in the evaluation of their effectiveness against more r…

• ExLipBaB: Exact Lipschitz Constant Computation for Piecewise Linear Neural Networks

  Tom A. Splittgerber · 18 de febrero de 2026

  It has been shown that a neural network's Lipschitz constant can be leveraged to derive robustness guarantees, to improve generalizability via regularization or even to construct invertible networks. Therefore, a number of methods varying in the tightness of their bounds and their computational cost…

• Unforgeable Watermarks for Language Models via Robust Signatures

  Huijia Lin, Kameron Shahabi, Min Jae Song · 18 de febrero de 2026

  Language models now routinely produce text that is difficult to distinguish from human writing, raising the need for robust tools to verify content provenance. Watermarking has emerged as a promising countermeasure, with existing work largely focused on model quality preservation and robust detectio…

• ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models

  Mitchell Piehl, Zhaohan Xi, Zuobin Xiong, Pan He, Muchao Ye · 18 de febrero de 2026

  Large language models (LLMs) are increasingly augmented with long-term memory systems to overcome finite context windows and enable persistent reasoning across interactions. However, recent research finds that LLMs become more vulnerable because memory provides extra attack surfaces. In this paper, …

• Protecting Language Models Against Unauthorized Distillation through Trace Rewriting

  Xinhang Ma, William Yeoh, Ning Zhang, Yevgeniy Vorobeychik · 18 de febrero de 2026

  Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate m…

• Exploiting Layer-Specific Vulnerabilities to Backdoor Attack in Federated Learning

  Mohammad Hadi Foroughi, Seyed Hamed Rastegar, Mohammad Sabokrou, Ahmad Khonsari · 18 de febrero de 2026

  Federated learning (FL) enables distributed model training across edge devices while preserving data locality. This decentralized approach has emerged as a promising solution for collaborative learning on sensitive user data, effectively addressing the longstanding privacy concerns inherent in centr…

• Complex-Valued Unitary Representations as Classification Heads for Improved Uncertainty Quantification in Deep Neural Networks

  Akbar Anbar Jafari, Cagri Ozcinar, Gholamreza Anbarjafari · 18 de febrero de 2026

  Modern deep neural networks achieve high predictive accuracy but remain poorly calibrated: their confidence scores do not reliably reflect the true probability of correctness. We propose a quantum-inspired classification head architecture that projects backbone features into a complex-valued Hilbert…

• The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety

  Max Springer, Chung Peng Lee, Blossom Metevier, Jane Castleman, Bohdan Turbal, Hayoung Jung, Zeyu Shen, Aleksandra Korolova · 18 de febrero de 2026

  Fine-tuning aligned language models on benign tasks unpredictably degrades safety guardrails, even when training data contains no harmful content and developers have no adversarial intent. We show that the prevailing explanation, that fine-tuning updates should be orthogonal to safety-critical direc…

• Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

  Kyle O'Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman · 18 de febrero de 2026

  Open-weight AI systems offer unique benefits, including enhanced transparency, open research, and decentralized access. However, they are vulnerable to tampering attacks which can efficiently elicit harmful behaviors by modifying weights or activations. Currently, there is not yet a robust science o…

• Closing the Distribution Gap in Adversarial Training for LLMs

  Chengzhi Hu, Jonas Dornbusch, David L\"udke, Stephan G\"unnemann, Leo Schwinn · 18 de febrero de 2026

  Adversarial training for LLMs is one of the most promising methods to reliably improve robustness against adversaries. However, despite significant progress, models remain vulnerable to simple in-distribution exploits, such as rewriting prompts in the past tense or translating them into other langua…

• SecCodeBench-V2 Technical Report

  Longfei Chen, Ji Zhao, Lanxiao Cui, Tong Su, Xingbo Pan, Ziyang Li, Yongxing Wu, Qijiang Cao, Qiyao Cai, Jing Zhang, Yuandong Ni, Junyao He, Zeyu Zhang, Chao Ge, Xuhuai Lu, Zeyu Gao, Yuxin Cui, Weisen Chen, Yuxuan Peng, Shengping Wang, Qi Li, Yukai Huang, Yukun Liu, Tuo Zhou, Terry Yue Zhuo, Junyang Lin, Chao Zhang · 18 de febrero de 2026

  We introduce SecCodeBench-V2, a publicly released benchmark for evaluating Large Language Model (LLM) copilots' capabilities of generating secure code. SecCodeBench-V2 comprises 98 generation and fix scenarios derived from Alibaba Group's industrial productions, where the underlying security issues …

• Approximation Theory for Lipschitz Continuous Transformers

  Takashi Furuya, Davide Murari, Carola-Bibiane Sch\"onlieb · 18 de febrero de 2026

  Stability and robustness are critical for deploying Transformers in safety-sensitive settings. A principled way to enforce such behavior is to constrain the model's Lipschitz constant. However, approximation-theoretic guarantees for architectures that explicitly preserve Lipschitz continuity have ye…

• Weight space Detection of Backdoors in LoRA Adapters

  David Puertolas Merenciano, Ekaterina Vasyagina, Raghav Dixit, Kevin Zhu, Ruizhe Li, Javier Ferrando, Maheep Chaudhary · 18 de febrero de 2026

  LoRA adapters let users fine-tune large language models (LLMs) efficiently. However, LoRA adapters are shared through open repositories like Hugging Face Hub \citep{huggingface_hub_docs}, making them vulnerable to backdoor attacks. Current detection methods require running the model with test input …

• The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes

  Mohammad Taufeeque, Stefan Heimersheim, Adam Gleave, Chris Cundy · 18 de febrero de 2026

  Training against white-box deception detectors has been proposed as a way to make AI systems honest. However, such training risks models learning to obfuscate their deception to evade the detector. Prior work has studied obfuscation only in artificial settings where models were directly rewarded for…