Adversarial Robustness in Machine Learning

Últimos artículos

• Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectories

  Dongcheng Zhang, Yi Zhang, Yuxin Chen, An Zhang, Xiang Wang, Chaochao Lu · 12 de mayo de 2026

  Large Reasoning Models possess remarkable capabilities for self-correction in general domain; however, they frequently struggle to recover from unsafe reasoning trajectories under adversarial attacks. Existing alignment methods attempt to mitigate this vulnerability by fine-tuning the model on exper…

• Research on Security Enhancement Methods for Adversarial Robust Large Language Model Intelligent Agents for Medical Decision-Making Tasks

  Saisai Hu · 12 de mayo de 2026

  Motivated by the challenge to improve the adversarial robustness, security, and trust of medical decision making intelligent agents, this study develops a full-link security enhancement framework, which describes "input risk perception - medical evidence constraint - knowledge consistency verificati…

• NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claims

  Varad Vishwarupe, Nigel Shadbolt, Marina Jirotka, Ivan Flechais · 12 de mayo de 2026

  Frontier AI safety claims - published assertions that a highly capable general-purpose model is below a threshold of concern, adequately mitigated, or suitable for release - increasingly shape model deployment, governance, and public trust. Yet the artefacts needed to evaluate them are routinely wit…

• Mental Health AI Safety Claims Must Preserve Temporal Evidence

  Srimonti Dutta, Ratna Kandala · 12 de mayo de 2026

  The safety of mental health AI is often judged at the wrong temporal scale. Current evaluations typically score isolated responses, endpoint outcomes, or aggregate dialogue quality, while clinically consequential failures may arise from the order and accumulation of interactions themselves, includin…

• From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

  Pedro Conde, Henrique Branquinho, Valerio Mazzone, Bruno Mendes, Andr\'e Baptista, Nuno Moniz · 12 de mayo de 2026

  AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perform best in real-world targets. Existing evaluation protocols assess and optimize for predefined goals such as capture-the-flag, remote code execution…

• A Robust Out-of-Distribution Detection Framework via Synergistic Smoothing

  Maria Stoica, Abdelrahman Hekal, Alessio Lomuscio · 12 de mayo de 2026

  Reliable out-of-distribution (OOD) detection is a critical requirement for the safe deployment of machine learning systems. Despite recent progress, state-of-the-art OOD detectors are highly susceptible to adversarial attacks, which undermines their trustworthiness in automated systems. To address t…

• Control Your View: High-Resolution Global Semantic Manipulation in Learned Image Compression

  Jiaming Liang, Chi-Man Pun, Weisi Lin, Greta Seng Peng Mok · 12 de mayo de 2026

  Learned image compression (LIC) integrates deep neural networks (DNNs) to map high-dimensional images into compact latent representations, reducing redundancy and achieving superior rate-distortion (RD) performance in benign settings. Unfortunately, due to inherent vulnerabilities in DNNs, LIC syste…

• The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring

  Ismail Hossain, Tanzim Ahad, Md Jahangir Alam, Sai Puppala, Syed Bahauddin Alam, Sajedul Talukder · 12 de mayo de 2026

  Jailbreak attacks -- adversarial prompts that bypass LLM alignment through purely linguistic manipulation -- pose a growing operational security threat, yet the field lacks large-scale, reproducible infrastructure for generating, categorizing, and evaluating them systematically. This paper addresses…

• AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems

  Boxuan Zhang, Jianing Zhu, Zeru Shi, Dongfang Liu, Ruixiang Tang · 12 de mayo de 2026

  LLM-based multi-agent systems are increasingly deployed on long-horizon tasks, but a single decisive error is often accepted by downstream agents and cascades into trajectory-level failure. Existing work frames this as \emph{post-hoc failure attribution}, diagnosing the responsible agent and step af…

• Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking

  Zhida He, Xiaoyu Wen, Han Qi, Ziyuan Zhou, Peng Yu, Xingcheng Xu, Dongrui Liu, Xia Hu, Chaochao Lu, Qiaosheng Zhang · 12 de mayo de 2026

  Deploying LLMs in multi-turn dialogues facilitates jailbreak attacks that distribute harmful intent across seemingly benign turns. Recent training-based multi-turn jailbreak methods learn long-horizon attack strategies from interaction feedback, but often rely on coarse trajectory-level outcome sign…

• NEXUS: Continual Learning of Symbolic Constraints for Safe and Robust Embodied Planning

  Tiehan Cui, Peipei Liu, Yanxu Mao, Congying Liu, Mingzhe Xing, Datao You · 12 de mayo de 2026

  While Large Language Models (LLMs) have catalyzed progress in embodied intelligence, a fundamental gap between their inherent probabilistic uncertainty and the strict determinism and verifiable safety required in the physical world. To mitigate this gap, this paper introduces NEXUS, a modular framew…

• Strategic commitments shape collective cybersecurity under AI inequality

  Adeela Bashir, Zia Ush Shamszaman, Zhao Song, The Anh Han · 12 de mayo de 2026

  The growing integration of AI into cybersecurity is reshaping the balance between attackers and defenders. When access to advanced AI-enabled defence tools is uneven, resource-limited defenders may be unable to adopt effective protection, creating persistent system vulnerabilities. We study the impa…

• Understanding Asynchronous Inference Methods for Vision-Language-Action Models

  Ayoub Agouzoul · 12 de mayo de 2026

  Vision-Language-Action (VLA) models offer a promising path to generalist robot control, but their inference latency causes observation staleness when generated actions are executed asynchronously. Several methods have been proposed concurrently to mitigate this problem: inference-time inpainting (IT…

• LLM-Agnostic Semantic Representation Attack

  Jiawei Lian, Jianhong Pan, Lefan Wang, Yi Wang, Tairan Huang, Shaohui Mei, Lap-Pui Chau · 12 de mayo de 2026

  Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs. Despite these safeguards, attackers can circumvent them by crafting adversarial prompts. Predominant token-level optimization methods primarily rely on optimizing for exact affirmative templates (e.g., …

• Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability

  Harsh Raj, Niranjan Orkat, Suvrorup Mukherjee, Aritra Guha, Cheryl Flynn, Subhabrata Majumdar · 12 de mayo de 2026

  This paper establishes a rigorous measurement science for AI agent reliability, providing a foundational framework for quantifying consistency under semantically preserving perturbations. By leveraging $U$-statistics for output-level reliability and kernel-based metrics for trajectory-level stabilit…

• Interactive Critique-Revision Training for Reliable Structured LLM Generation

  Fei Xu Yu, Zuyuan Zhang, Mahdi Imani, Nathaniel D. Bastian, Tian Lan · 12 de mayo de 2026

  In structured decision-making workflows such as form filling, compliance checking, and maintenance reporting, LLM outputs must be locally correct, globally consistent, and auditable against task-specific rules. Existing refinement methods often rely on heuristic debate, self-play, or LLM-generated s…

• Trapping Attacker in Dilemma: Examining Internal Correlations and External Influences of Trigger for Defending GNN Backdoors

  Fan Yang, Binyan Xu, Di Tang, Kehuan Zhang · 12 de mayo de 2026

  GNNs have become a standard tool for learning on relational data, yet they remain highly vulnerable to backdoor attacks. Prior defenses often depend on inspecting specific subgraph patterns or node features, and thus can be circumvented by adaptive attackers. We propose PRAETORIAN, a new defense tha…

• PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelines

  Riya Tapwal, Abhishek Kumar, Carsten Maple · 12 de mayo de 2026

  Multi-agent LLM systems introduce a security risk in which sensitive information accessed by one agent can propagate through shared context and reappear in downstream outputs, even without explicit adversarial intent. We formalise this phenomenon as propagation amplification, where leakage risk incr…

• Internalizing Safety Understanding in Large Reasoning Models via Verification

  Yi Zhang, Yuxin Chen, Leheng Sheng, Dongcheng Zhang, Chaochao Lu, Xiang Wang, An Zhang · 12 de mayo de 2026

  While explicit Chain-of-Thought (CoT) empowers large reasoning models (LRMs), it enables the generation of riskier final answers. Current alignment paradigms primarily rely on externally enforced compliance, optimizing models to detect malicious prompts rather than evaluating the safety of their own…

• ASIA: an Autonomous System Identification Agent

  Dario Piga, Marco Forgione · 12 de mayo de 2026

  Over the years, research in system identification has provided a rich set of methods for learning dynamical models, together with well-established theoretical guarantees. In practice, however, the choice of model class, training algorithm, and hyperparameter tuning is still largely left to empirical…

• Sanity Checks for Long-Form Hallucination Detection

  Geigh Zollicoffer, Minh Vu, Hongli Zhan, Raymond Li, Manish Bhattarai · 12 de mayo de 2026

  Hallucination detection methods for large language models increasingly operate on chain-of-thought reasoning traces, yet it remains unclear whether they evaluate the reasoning itself or merely exploit surface correlates of the final answer. We introduce a controlled-invariance methodology that expos…

• The Attacker in the Mirror: Breaking Self-Consistency in Safety via Anchored Bipolicy Self-Play

  Gabriele La Malfa, Emanuele La Malfa, Saar Cohen, Jie M. Zhang, Michael Luck, Michael Wooldridge, Elizabeth Black · 12 de mayo de 2026

  Self-play red team is an established approach to improving AI safety in which different instances of the same model play attacker and defender roles in a zero-sum game, i.e., where the attacker tries to jailbreak the defender; if self-play converges to a Nash equilibrium, the model is guaranteed to …

• Latent Personality Alignment: Improving Harmlessness Without Mentioning Harms

  Linh Le, David Williams-King, Mohamed Amine Merzouk, Aton Kamanda, Adam Oberman · 12 de mayo de 2026

  Current adversarial robustness methods for large language models require extensive datasets of harmful prompts (thousands to hundreds of thousands of examples), yet remain vulnerable to novel attack vectors and distributional shifts. We propose Latent Personality Alignment (LPA), a sample-efficient …

• When (and How) to Trust the Expert: Diagnosing Query-Time Expert-Guided Reinforcement Learning

  Yann Berthelot, Philippe Preux, Riad Akrour · 12 de mayo de 2026

  Many continuous-control problems ship with a competent but suboptimal controller (a tuned PID, a hand-designed gait). A growing family of methods uses such controllers as queryable experts during RL, but each method has been proposed in isolation, on a different benchmark, without imperfect-expert t…

• Insider Attacks in Multi-Agent LLM Consensus Systems

  Xiaolin Sun, Zixuan Liu, Yibin Hu, Zizhan Zheng · 12 de mayo de 2026

  Large language models (LLMs) are increasingly deployed in multi-agent systems where agents communicate in natural language to solve tasks jointly. A key capability in such systems is consensus formation, where agents iteratively exchange messages and update decisions to reach a shared outcome. Howev…