Adversarial Robustness in Machine Learning

Neueste Paper

• Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation

  Alexandre Cristov\~ao Maiorano · 24. Juli 2026

  Production LLM applications commonly stack a regex filter in front of model-side alignment; prior work found no measurable coverage gain from adding a live Gemini backend behind an active regex filter. We ask whether that ceiling holds when the corpus is \emph{designed to bypass the regex}. We intro…

• Hilbert Operator for Progressive Encoding (HOPE): A Mathematical Framework for Deconstructing Learned Representations in Deep Networks

  Hossein Mobahi, Peter L. Bartlett · 24. Juli 2026

  Deep neural networks encode complex representations, but deconstructing this internal knowledge remains a challenge. Given the link between learning and compression, network compression offers a promising lens to analyze this knowledge. However, standard compression heuristics often suffer from scal…

• OpenForgeRL: Train Harness-native Agents in Any Environment

  Xiao Yu, Baolin Peng, Ruize Xu, Hao Zou, Qianhui Wu, Hao Cheng, Wenlin Yao, Nikhil Singh, Zhou Yu, Jianfeng Gao · 24. Juli 2026

  Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks can…

• V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure

  Zhetong Zhang, Honghao Fu, Miao Xu, Yiwei Wang, Yujun Cai · 24. Juli 2026

  As Video Large Language Models are increasingly deployed in real-world applications, ensuring their safety alignment has become critical. Counterintuitively, we find that harmful videos paired with benign queries achieve higher attack success rates than the same videos paired with explicitly harmful…

• From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

  Muntasir Adnan, Manile Srun, Carlos C. N. Kuhn · 24. Juli 2026

  The original ALPHA benchmark introduced a taxonomy-aware penalty for evaluating CWE-level vulnerability prediction in Python and proposed that the penalty could theoretically also serve as a training signal. This paper provides that validation. We compare three delivery mechanisms: supervised fine-t…

• Detecting Neural Network Failures through Spectral Analysis of Internal Activations

  Arunan J · 24. Juli 2026

  Neural network misclassifications exhibit characteristic spectral instability in internal activations that is invisible at the output layer. This phenomenon is identified and formalized as Spectral Drift -- the frequency-domain distance between consecutive layer activations -- with empirical validat…

• Robust Critics: Defending LLMs Against Multi-Turn Attacks

  Roman Belaire, Arunesh Sinha, Pradeep Varakantham · 24. Juli 2026

  When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question? This ambiguity is one of the central challenges of LLM safety. A model that assumes the worst harms legitimate users; one that assumes the best is easily exploited. The problem i…

• GuardianAgentBench: Where Agents Fail and How to Guard Them

  Vishal Ishwar Naik, Chenyu Xu, Donna Dong, Hussein Hassan, Abhishek Pradhan, Ofer Mendelevitch, Tallat Shafat, Humayun Irshad · 24. Juli 2026

  As large language model agents increasingly operate autonomously with access to tools and external environments, ensuring their safe and reliable behavior becomes critical. We present GuardianAgentBench (GABench), a benchmark of 580 scenarios across six domains evaluated on three production-ready fr…

• DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions

  Zishan Shao, Lixun Zhang, Kangning Cui, Yixiao Wang, Ting Jiang, Hancheng Ye, Qinsi Wang, Zhixu Du, Yuzhe Fu, Fan Yang, Danyang Zhuo, Yiran Chen, Hai Helen Li · 24. Juli 2026

  Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time rather than during prefill. We propose DecodeShare, a protocol that identifies a low-dimensional subspace consistently sh…

• Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations

  Xinyue Fang, Zhiliang Tian, Zhen Huang, Ziyi Pan, Zhihua Wen, Xi Wang, Quntian Fang, Dongsheng Li · 24. Juli 2026

  Existing LLM hallucination mitigation methods, including prompt engineering and model optimization, either hardly alter models'internal knowledge or have poor cross-domain generalization. Contrastive decoding mitigates hallucinations by using layer-wise differences in LLMs. However, prior studies on…

• Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility

  Fan Li, Mengting Pan, Sijia Xu, Xiaoyang Wang, Chen Chen, Wenjie Zhang · 24. Juli 2026

  Proprietary large language models (LLMs) entail substantial intellectual and financial investment, making them valuable intellectual property (IP). However, even when deployed via black-box APIs, these models remain vulnerable to unauthorized knowledge distillation, which allows adversaries to cheap…

• Making Open-Source Text LLM Watermarks Durable Against Merging

  Luisa Scharff, Thibaud Gloaguen, Robin Staab, Martin Vechev · 24. Juli 2026

  Open-source LLMs (OSMs)arereaching near state-of-the-art performance, prompting prior works to trace the text they generate by embedding text watermarking algorithms directly into their weights. Yet, OSMs are subject to post-training modifications, which has been shown to remove the watermark. Model…

• Geometric Configurations of Perturbed Jailbreak Prompts

  Lynn Delcon, Andres Algaba, Vincent Ginis · 24. Juli 2026

  Perturbation techniques that turn unsuccessful jailbreak prompts into successful ones are continuously evolving, constituting a major security threat to LLM safety. In this paper, we investigate the internal representations of such string-level perturbed jailbreak inputs in the small weight models o…

• Test Case Prioritization for DNNs via Neural Collapse Instability

  Chunyu Liu, Mingyuan Li, Yang Li, Wenmin Li, Fei Gao, Tengfei Tu, Su-Juan Qin · 23. Juli 2026

  With the widespread deployment of deep neural networks (DNNs) in safety-critical domains, reducing the cost of model validation under limited testing budgets has become increasingly important. Existing test case prioritization techniques often rely on single-checkpoint confidence signals derived fro…

• Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

  Sen Yang, Yuen-Hei Yeung · 23. Juli 2026

  Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes. In a controlled nonce-fact testbed with a matched retraining reference, we find this criterion can favor methods that retain held-out knowledge: candidates it rates adequate score held-out forget facts $-2.82$…

• Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents

  Or Zion Eliav, Eyal Lenga, Shir Bernstien, Yisroel Mirsky · 23. Juli 2026

  Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment. We formalize agent reconnaissance by modeling the process and identifying the knowledge assets it seeks to extr…

• HijackKV: New Threat in Position-Independent KV Cache Reuse

  Yichi Zhang, Zhiqi Wang, Huan Zhang, Yuchen Yang · 23. Juli 2026

  Key-Value (KV) cache reduces inference latency in large language models (LLMs). Traditional prefix-based reuse has low cache hit rates across inference requests because it requires exact token and position matches. To improve efficiency, recent system optimizations introduce position-independent KV …

• Harnessing Disagreement: Detecting Correlated Agreement Blindness in Multi-Agent Triage

  Shay Seiya McDonnell, Avantika Singh, Quoc-Viet Pham, Vratislav Havlik, Gregory M. P. O'Hare · 23. Juli 2026

  Disagreement-triggered escalation can create a structural blind spot in multi-agent arbitration: as base learners improve, they tend to converge, weakening safety monitoring where correlated failures concentrate. We term this correlated agreement blindness and present ARAT (Arbitrated Reasoning Agen…

• Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

  Luca Scionis, Luca Melis, Maura Pintor, Fabio Brau, Ambra Demontis, Giorgio Fumera, Fabio Roli, Battista Biggio · 23. Juli 2026

  Adversarial robustness is commonly evaluated with predefined attack ensembles, such as AutoAttack, at a single perturbation budget $\varepsilon$ and on a selective choice of perturbation norms. We argue this formulation is fundamentally limited. First, robustness--perturbation curves may intersect o…

• Data-Poisoning Audits for Causal Effect Estimation

  Kwangho Kim · 23. Juli 2026

  Observational causal analyses increasingly pool records across sites, vendors, and collection systems, creating vulnerability to append-only attacks in which plausible records are strategically selected to alter a reported treatment effect. We develop a data-poisoning audit for augmented inverse-pro…

• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model

  Sibo Wang, Jie Zhang, Shiguang Shan, Xilin Chen, Wen Gao · 22. Juli 2026

  While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks, posing significant security risks. Existing defense methods predominantly target single-task scenarios (e.g., zero-sho…

• The Economics of Autonomy: Real-Time Risk Indexing for Insurable AI-Driven 6G Systems

  Anthony Kiggundu, Michael Zentarra, Christoph Lipps, Hans D. Schotten · 22. Juli 2026

  The transition to sixth-generation (6G) networks transforms wireless infrastructure into a cognitive substrate supporting Vehicle-to-Everything (V2X), Industrial IoT (IIoT), and Integrated Sensing and Communication (ISAC). In this paradigm, autonomous agentic AI performs orchestration at millisecond…

• SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement

  Arther Tian, Alex Ding, Simon Wu, Aaron Chan · 22. Juli 2026

  Procuring supervised fine-tuning (SFT) data forces a buyer to decide, before any downstream training, whether a candidate corpus is worth acquiring. We present \sys{}, a statistics-first gating architecture that treats procurement as a cost-aware routing problem over three intrinsic quality axes -- …

• FALCON-Discover: Discovering Concentrated False-Confidence Regions for Calibration

  Filippo Cenacchi, Longbing Cao, Runze Yang · 22. Juli 2026

  Calibration is usually evaluated in aggregate, but the most dangerous failures are often local: predictions that remain highly confident despite being wrong. We study this failure mode as false-confidence concentration, the extent to which confident errors occupy compact, discoverable regions of pre…

• ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D

  Lena Libon, Ben Rank, Jehyeok Yeon, David Schmotz, Jeremy Qin, Daniel Donnelly, Derck Prinzhorn, Maksym Andriushchenko · 22. Juli 2026

  As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such approach: rather than trusting the agent, it treats it as a potential adversary and uses a monitor to detect covert sab…