Adversarial Robustness in Machine Learning

Neueste Paper

• Behavioral Integrity Verification for AI Agent Skills

  Yuhao Wu, Tung-Ling Li, Hongliang Liu · 13. Mai 2026

  Agent skills extend LLM agents with privileged third-party capabilities such as filesystem access, credentials, network calls, and shell execution. Existing safety work catches malicious prompts and risky runtime actions, but the skill artifact itself goes unverified. We formalize this as the behavi…

• Robust Multi-Agent Path Finding under Observation Attacks: A Principled Adversarial-Plus-Smoothing Training Recipe

  Riad Ahmed · 13. Mai 2026

  Decentralized multi-agent path finding (MAPF) routes a team of agents on a shared grid, each acting from its own local view. The standard solution trains one shared neural policy with Proximal Policy Optimization (PPO), a popular on-policy reinforcement learning algorithm. Such a policy works well o…

• AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelines

  Tingxi Li, Mingfang Ji, Ravishka Shemal Rathnasuriya, Simin Chen, Yitao Hu, Wei Yang · 13. Mai 2026

  Modern machine learning deployments increasingly compose specialized models into dynamic inference pipelines, where upstream components produce intermediate predictions that determine the workload and inputs of downstream components. The cost of processing an input is therefore not determined by any…

• When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Models

  Ziyu Liu, Tao Li, Tianjie Ni, Xiaolong Lan, Wengang Ma, Tao Yang, Guohua Wang, Junjiang He · 13. Mai 2026

  Backdoor vulnerabilities widely exist in the fine-tuning of large language models(LLMs). Most backdoor poisoning methods operate mainly at the token level and lack deeper semantic manipulation, which limits stealthiness. In addition, Prior attacks rely on a single fixed trigger to induce harmful out…

• RACC: Representation-Aware Coverage Criteria for LLM Safety Testing

  Zeming Wei, Zhixin Zhang, Chengcan Wu, Yihao Zhang, Xiaokun Luan, Meng Sun · 13. Mai 2026

  Large Language Models (LLMs) face severe safety risks from jailbreak attacks, yet current safety testing largely relies on static datasets and lacks systematic criteria to evaluate test suite quality and adequacy. While coverage criteria have proven effective for smaller neural networks, they are im…

• Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection

  Shreen Gul, Mohamed Elmahallawy, Ardhendu Tripathy, Sanjay Madria · 13. Mai 2026

  Deep learning models are increasingly deployed in safety-critical applications, where reliable out-of-distribution (OOD) detection is essential to ensure robustness. Existing methods predominantly rely on the penultimate-layer activations of neural networks, assuming they encapsulate the most inform…

• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment

  Bo Yin, Qi Li, Xinchao Wang · 13. Mai 2026

  Tool-using LLM agents fail through trajectories rather than only final responses, as they may execute unsafe tool calls, follow injected instructions, comply with harmful requests, or over-refuse benign tasks despite producing a seemingly safe answer. Existing safety-alignment signals are largely re…

• Reward Hacking in Rubric-Based Reinforcement Learning

  Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang, Anisha Gunjal, Bing Liu, Yunzhong He · 13. Mai 2026

  Reinforcement learning with verifiable rewards has enabled strong post-training gains in domains such as math and coding, though many open-ended settings rely on rubric-based rewards. We study reward hacking in rubric-based RL, where a policy is optimized against a training verifier but evaluated ag…

• Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw

  Hongwei Yao, Yiming Liu, Yiling He, Bingrun Yang · 13. Mai 2026

  Agentic language-model systems increasingly rely on mutable execution contexts, including files, memory, tools, skills, and auxiliary artifacts, creating security risks beyond explicit user prompts. This paper presents DeepTrap, an automated framework for discovering contextual vulnerabilities in Op…

• SoK: Unlearnability and Unlearning for Model Dememorization

  Mengying Zhang, Derui Wang, Ruoxi Sun, Xiaoyu Xia, Shuang Hao, Minhui Xue · 13. Mai 2026

  Advanced model dememorization methods, including availability poisoning (unlearnability) and machine unlearning, are emerging as key safeguards against data misuse in machine learning (ML). At the training stage, unlearnability embeds imperceptible perturbations into data before release to reduce le…

• Scalable Token-Level Hallucination Detection in Large Language Models

  Rui Min, Tianyu Pang, Chao Du, Minhao Cheng, Yi R. Fung · 13. Mai 2026

  Large language models (LLMs) have demonstrated remarkable capabilities, but they still frequently produce hallucinations. These hallucinations are difficult to detect in reasoning-intensive tasks, where the content appears coherent but contains errors like logical flaws and unreliable intermediate r…

• Can a Single Message Paralyze the AI Infrastructure? The Rise of AbO-DDoS Attacks through Targeted Mobius Injection

  Zi Liang, Ronghua Li, Yanyun Wang, Qingqing Ye, Haibo Hu · 13. Mai 2026

  Large Language Model (LLM) agents have emerged as key intermediaries, orchestrating complex interactions between human users and a wide range of digital services and LLM infrastructures. While prior research has extensively examined the security of LLMs and agents in isolation, the systemic risk of …

• Action Hallucination in Generative Vision-Language-Action Models

  Harold Soh, Eugene Lim · 13. Mai 2026

  Robot Foundation Models, such as VLAs, promise end-to-end generative robot policies with broad generalization. Yet it remains unclear whether they fundamentally resolve the core problem of action generation in embodied settings, or overcome the long-standing challenges of robotics. We address this q…

• Native Explainability for Bayesian Confidence Propagation Neural Networks: A Framework for Trusted Brain-Like AI

  Georgios Makridis, Georgios Fatouros, John Soldatos, George Katsis, Dimosthenis Kyriazis · 13. Mai 2026

  The EU Artificial Intelligence Act (Regulation 2024/1689), fully applicable to high-risk systems from August 2026, creates urgent demand for AI architectures that are simultaneously trustworthy, transparent, and feasible to deploy on resource-constrained edge devices. Brain-like neural networks buil…

• The Scaling Law of Evaluation Failure: Why Simple Averaging Collapses Under Data Sparsity and Item Difficulty Gaps, and How Item Response Theory Recovers Ground Truth Across Domains

  Jung Min Kang · 13. Mai 2026

  Benchmark evaluation across AI and safety-critical domains overwhelmingly relies on simple averaging. We demonstrate that this practice produces substantially misleading rankings when two conditions co-occur: (1) the evaluation matrix is sparse and (2) items vary substantially in difficulty. Through…

• Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights

  Wenbo Chen, Veena Padmanabhan, Tootiya Giyahchi, Elaine Wong, Leman Akoglu · 13. Mai 2026

  Hallucination, broadly referring to unfaithful, fabricated, or inconsistent content generated by LLMs, has wide-ranging implications. Therefore, a large body of effort has been devoted to detecting LLM hallucinations, as well as designing benchmark datasets for evaluating these detectors. In this wo…

• Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization

  Navid Rezazadeh, Arash Gholami Davoodi · 13. Mai 2026

  Certified verification of transformer attention requires bounding the softmax function over interval constraints on the pre-softmax scores. Existing verifiers relax softmax ndependently of the downstream objective, leaving avoidable slack. We prove that the exact optimum of this score-box problem is…

• Classifier Context Rot: Monitor Performance Degrades with Context Length

  Sam Martin, Fabien Roger · 13. Mai 2026

  Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely contain transcripts longer than 100K tokens. We show that when used as classifiers, current frontier models fail to notic…

• BSO: Safety Alignment Is Density Ratio Matching

  Tien-Phat Nguyen, Truong Nguyen, Thin Nguyen, Duy Minh Ho Nguyen, Ngoc-Thanh Dinh, Trung Le · 13. Mai 2026

  Aligning language models for both helpfulness and safety typically requires complex pipelines-separate reward and cost models, online reinforcement learning, and primal-dual updates. Recent direct preference optimization approaches simplify training but incorporate safety through ad-hoc modification…

• Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs

  Sangyeon Yoon, Wonje Jeung, Yoonjun Cho, Dongjae Jeon, Albert No · 13. Mai 2026

  Fine-tuning APIs make frontier LLMs easy to customize, but they can also weaken safety alignment during fine-tuning. While prior work shows that benign supervised fine-tuning (SFT) can reduce refusal behavior, deployed fine-tuning pipelines increasingly support preference-based objectives, whose saf…

• Leveraging RAG for Training-Free Alignment of LLMs

  John T. Halloran · 13. Mai 2026

  Large language model (LLM) alignment algorithms typically consist of post-training over preference pairs. While such algorithms are widely used to enable safety guardrails and align LLMs with general human preferences, we show that state-of-the-art alignment algorithms require significant computatio…

• Localising Dropout Variance in Twin Networks

  Cooper Doyle · 13. Mai 2026

  Accurate individual treatment-effect estimation demands not only reliable point predictions but also uncertainty measures that help practitioners \emph{locate} the source of model failure. We introduce a layer-wise variance decomposition for deep twin-network models: by toggling Monte Carlo Dropout …

• SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces

  Chang Jin, An Wang, Zeming Wei, Kai Wang, Biaojie Zeng, Qiaosheng Zhang, Chao Yang, Jingjing Qu, Xia Hu, Xingcheng Xu · 13. Mai 2026

  Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even w…

• SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models

  Xinyi Zeng, Xue Yang, Jingyuan Zhang, Huanqian Yan, Xiang Chen, Kaiwen Wei, Hankun Kang, Yu Tian · 13. Mai 2026

  Multimodal large language models (MLLMs) are gaining increasing attention. Due to the heterogeneity of their input features, they face significant challenges in terms of jailbreak defenses. Current defense methods rely on costly fine-tuning or inefficient post-hoc interventions, limiting their abili…

• Partial Model Sharing Improves Byzantine Resilience in Federated Conformal Prediction

  Ehsan Lari, Reza Arablouei, Stefan Werner · 13. Mai 2026

  We propose a Byzantine-resilient federated conformal prediction (FCP) method that leverages partial model sharing, where only a subset of model parameters is exchanged each round. Unlike existing robust FCP approaches that primarily harden the calibration stage, our method protects both the federate…