Adversarial Robustness in Machine Learning

Últimos artículos

• BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks

  Ivan Saboli\'c, Marin Or\v{s}i\'c, Josip \v{S}ari\'c, Sven Lon\v{c}ari\'c · 3 de junio de 2026

  Supervised fine-tuning is the predominant approach for adapting autoregressive vision-language models to downstream tasks. Recent work has shown that this paradigm is highly vulnerable to backdoor attacks, and that existing defenses are ineffective in open-ended generation settings. In response, we …

• PURGE: Projected Unlearning via Retain-Guided Erasure

  Vedant Jawandhia, Daksh Ahuja, Ghufran Alam Siddiqui, Prashant Trivedi, Yash Sinha, Pratik Narang · 3 de junio de 2026

  We propose PURGE, a machine unlearning algorithm built on a simple but an under-exploited observation: continual learning (CL) and machine unlearning (MU) which are fundamentally dual problems. CL tries to learn new tasks without forgetting old ones; MU tries to erase specific data without hurting r…

• P\textsuperscript{2}-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

  Ruipeng Zhang, Zhihao Li, Haozhang Yuan, C. L. Philip Chen, Tong Zhang · 3 de junio de 2026

  Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs). Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans, thereby addressing the hallucination issue. Despite its success, this paradigm ha…

• Outsmarting the Chameleon: Counterfactual Decoupling for Tactical OOD Shifts in Live Streaming Risk Assessment

  Yiran Qiao, Jing Chen, Jiaqi Xu, Yang Liu, Qiwei Zhong, Xiang Ao · 3 de junio de 2026

  Live streaming has emerged as a primary medium for social interaction and digital commerce, yet it is increasingly plagued by sophisticated risks. A fundamental challenge in this domain is \emph{tactical out-of-distribution (OOD) shift}: while malicious actors maintain stable underlying objectives, …

• SAIL: Sound Abstract Interpreters with LLMs

  Qiuhan Gu, Avaljot Singh, Gagandeep Singh · 3 de junio de 2026

  How to construct globally sound abstract interpreters to safely approximate program behaviors remains a bottleneck in abstract interpretation. In this paper, we show the potential of using state-of-the-art LLMs to automate this tedious process. Focusing on the neural network verification area, we sy…

• AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses

  Maxime Schwarzer, Johannes F. Loevenich, Gustavo S\'anchez, Laurin Holz, Thies M\"ohlenhof, Tobias H\"urten, Roberto Rigolin F. Lopes, Veit Hagenmeyer · 3 de junio de 2026

  Ensuring the protection of Artificial Intelligence (AI) models deployed in military Command and Control (C2) systems and critical infrastructure is essential for maintaining information superiority. Model Extraction Attacks (MEAs) pose a significant threat, as they enable adversaries to replicate pr…

• MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks an dClassifier-Based Defenses for Medical AI Safety

  Anushka Sheoran, Yiduo Hao · 3 de junio de 2026

  Patient-facing medical chatbots are commonly evaluated on single-turn prompts, yet real users push back after refusals, add urgency, and invoke authority. We introduce MultiTurnPSB, a four-turn adversarial extension of PatientSafetyBench, and evaluate GPT-4.1-mini under fixed template, template-adap…

• "**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems

  Hang Li, Fedor Filippov, Yuling Lin, Pengfei He, Kaiqi Yang, Yucheng Chu, Yingqian Cui, Hui Liu, Jiliang Tang · 3 de junio de 2026

  The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educators can deploy AG systems across diverse tasks using only na…

• Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs

  Vincent Limbach, Jonas Dornbusch, David L\"udke, Stephan G\"unnemann, Leo Schwinn · 3 de junio de 2026

  Accurately evaluating adversarial robustness is a longstanding challenge. A flawed attack design can inflate robustness estimates, making deployment risk assessment and defense comparison unreliable. Historically, standardized attacks such as AutoAttack have largely resolved this for image classifie…

• Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs

  Aizierjiang Aiersilan · 3 de junio de 2026

  We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest. Across three $7$B--$8$B instruction-tuned models (Llama-3.1-8B, Mistral-7B, Qwen2.5-7B) loaded in $4$-bit NF4 quantization, we extract …

• RogueMerge: Robust and Unified Attacks against LLM Model Merging

  Jinghuai Zhang, Yetian He, Kunlin Cai, Han Zhao, Fnu Suya, Yuan Tian · 3 de junio de 2026

  Model merging composes specialized capabilities into a single LLM by aggregating task vectors sourced from unverified public platforms, exposing a critical supply-chain attack surface: Because any malicious behavior can be encoded into a task vector, and merging grants third-party vectors direct wri…

• Position: Adversarial ML for LLMs Is Not Making Any Progress

  Javier Rando, Jie Zhang, Nicholas Carlini, Florian Tram\`er · 3 de junio de 2026

  In the past decade, considerable research effort has been devoted to securing machine learning (ML) models that operate in adversarial settings. Yet, progress has been slow even for simple "toy" problems (e.g., robustness to small adversarial perturbations) and is often hindered by non-rigorous eval…

• Post-Hoc Robustness for Model-Based Reinforcement Learning

  Siemen Herremans, Ali Anwar, Siegfried Mercelis · 3 de junio de 2026

  To improve the real-world applicability of reinforcement learning (RL), the field of adversarially robust RL studies how to train agents under adversarial environment perturbations. In this setting, a protagonist agent optimizes a policy under environmental perturbations from an adversary, resulting…

• Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing

  Alexandre Cristov\~ao Maiorano · 3 de junio de 2026

  Production LLM applications stack several defense families -- refusal-phrase filters, token-budget controls, model allowlists, rate limits, tool-registry authentication -- yet existing breach-and-attack-simulation (BAS) benchmarks report a single aggregate coverage number, hiding which family closes…

• TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches

  Zhengxian Huang, Wenjun Zhu, Haoxuan Qiu, Xiaoyu Ji, Wenyuan Xu · 3 de junio de 2026

  By integrating Chain-of-Thought (CoT) reasoning, Vision-Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation, particularly by improving generalization and interpretability. However, the security of CoT-based reasoning mechanisms remains largely unexplored. In th…

• Geometry-Aware Tabular Diffusion

  David Turtora Zagardo · 3 de junio de 2026

  Tabular synthesis is critical for privacy-preserving sharing and augmentation, yet diffusion models rely on implicit mechanisms to capture inter-column relationships. We introduce Geometry-Aware Tabular Diffusion (GATD), which augments tabular diffusion denoisers with pairwise angles and lengths com…

• D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting

  Huanli Gong, Zhipeng Wei, Yu Fu, Haz Sameen Shahgir, Ananya Gupta, Yue Dong, N. Benjamin Erichson · 3 de junio de 2026

  Multi-turn jailbreak attacks pose a growing threat to large language model (LLM) safety because they exploit feedback from auxiliary judge models to iteratively refine prompts toward harmful goals. Existing defenses largely detect or block unsafe content at individual turns or at the final response,…

• Patcher: Post-Hoc Patching of Backdoored Large Language Models

  Anjun Gao, Yueyang Quan, Yufei Xia, Zhuqing Liu, Minghong Fang · 3 de junio de 2026

  Large language models remain vulnerable to jailbreak backdoor attacks, where adversaries poison safety alignment data to embed hidden triggers that bypass safety mechanisms. Existing defenses often require comprehensive attack information or multiple triggered examples, making them impractical when …

• NeuroArmor: Safe-Variant-Guided Representation Consistency for Selective Re-Anchoring in Jailbreak Defense

  Zhongyang Lin, Ziran Zhao, Feifei Zhai, Pengyuan Liu · 3 de junio de 2026

  Large language models remain vulnerable to jailbreak attacks that hide harmful intent behind seemingly ordinary requests such as role-play, translation, encoding, adversarial suffixes, and multi-turn buildup. Existing defenses still struggle to handle these attacks without over-blocking benign but s…

• DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair

  Qinyan Zhou, Peixin Zhang, Jun Sun, Haonan Zhang, Dongxia Wang · 3 de junio de 2026

  While safety alignment and guardrails help large language models (LLMs) avoid harmful outputs, they can also induce overrefusal, i.e., unwarranted rejection of benign queries that merely appear risky. We present DDOR (Delta Debugging for OverRefusal), a fully automated and explainable framework for …

• Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments

  Deepak Kumar Panda, Adolfo Perrusquia, Weisi Guo · 3 de junio de 2026

  Autonomous unmanned aerial vehicles (UAVs) increasingly rely on reinforcement learning (RL) for navigation. However, global navigation satellite system (GNSS) spoofing attacks can induce out-of-distribution observation shifts that corrupt value estimation and degrade mission performance. Existing ro…

• Phantom Transfer: Data Poisoning can Survive Data-Level Defences

  Andrew Draganov, Tolga H. Dur, Anandmayi Bhongade, Mary Phuong · 3 de junio de 2026

  We present a data poisoning attack -- Phantom Transfer -- with the property that, even if you know precisely how the poison was placed into an otherwise benign dataset, you cannot filter it out. We achieve this by modifying subliminal learning to work in real-world contexts and demonstrate that the …

• Gate AI: LLM Security Benchmark Evaluation Methodology and Results

  Ryle Goehausen, Marcus Sousa · 3 de junio de 2026

  Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored acros…

• Easy-to-Use Shielding for Reinforcement Learning

  Stefan Pranger, Bettina K\"onighofer · 3 de junio de 2026

  Safe exploration is a key challenge in Reinforcement Learning (RL) that aims to prevent agents from making harmful decisions while exploring their environment. Safe exploration is a key challenge in Reinforcement Learning (RL) that aims to prevent agents from making harmful decisions while exploring…

• MESA: Improving MoE Safety Alignment via Decentralized Expertise

  Yitong Sun, Yao Huang, Teng Li, Ranjie Duan, Yichi Zhang, Xingjun Ma, Hui Xue, Xingxing Wei · 2 de junio de 2026

  Mixture-of-Experts (MoE) architectures scale Large Language Models (LLMs) efficiently, enabling greater capacity with reduced computational cost by dynamically routing inputs to relevant experts, yet introduce a critical vulnerability: Safety Sparsity, where safety capabilities concentrate in few ex…