Adversarial Robustness in Machine Learning

Últimos artículos

• Between Safe Boundaries: Exploiting Temporal Consistency for Jailbreaking Text-To-Video Generation Models

  Xingkai Peng, Jun Jiang, Jiayang Liu, Kejiang Chen, Weiming Zhang · 21 de julio de 2026

  Recently, text-to-video (T2V) models have been widely deployed, sparking growing concerns over their robustness against jailbreak attacks. Existing jailbreak methods, mostly adapted from text-to-image attacks, suffer notable drawbacks when applied to T2V systems. They fail to fully leverage temporal…

• Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models

  Dongdong Yang, Deyue Zhang, Zhao Liu, Zonghao Ying, Wenzhuo Xu, Jiankai Jin, Xiangzheng Zhang, Quanchen Zou · 21 de julio de 2026

  Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images. Most existing jailbreak attacks primarily rely on heuristic prompt engine…

• A Survey on the Verification of Reinforcement Learning Policies

  Luca Marzari, Ezio Bartocci, Enrico Marchesini · 21 de julio de 2026

  Reinforcement learning (RL) is increasingly applied in complex, safety-critical domains, yet the lack of rigorous behavioral guarantees for neural network-based policies remains a major barrier to deployment. Recent advances in policy expressiveness and scale have intensified this challenge, leading…

• ALLUDE: A Unified Evaluation System for Configurable Attacks in Differentiable Environments

  Mansi Phute, Alexander Greenhalgh, Matthew Hull, Haoran Wang, Alec Helbling, ShengYun Peng, Elliott Faa, Willian Lunardi, Martin Andreoni, Wenke Lee, Duen Horng Chau · 21 de julio de 2026

  Adversarial attacks against vision models like object detectors are often evaluated under limited conditions, leaving their performance under-characterized. Bridging simulation and differentiable rendering enables more robust, end-to-end evaluation of these adversarial attacks, yet there is no easy-…

• Certified Training for Convolutional Perturbations

  Benedikt Br\"uckner, Alessio Lomuscio · 21 de julio de 2026

  Vision models have been found to be susceptible to perturbations such as motion blur induced at runtime by a shaking camera. This impedes their deployment in critical applications since phenomena such as slightly blurred vision might lead to failures, for example an object detector missing objects. …

• Online learning of neural state-space models

  Bendeg\'uz Gy\"or\"ok, Tam\'as P\'eni, Maarten Schoukens, Roland T\'oth · 21 de julio de 2026

  Recent advances in deep-learning-based nonlinear system identification have led to encoder-based estimation of neural state-space (ANN-SS) models that achieve state-of-the-art performance in offline settings by estimating initial model states from past input-output data. These methods are typically …

• Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration

  Jie Hu · 21 de julio de 2026

  Test-time collaboration, including self-consistency, best-of-N selection, critic models, and verifier pipelines, is often credited with broadly improving LLM reasoning, yet its gains are uneven and sometimes negative. We ask when training-free collaboration should be expected to help. For a fixed ca…

• Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI

  Bogdan Raduta, Horia Velicu, Alexandru Preda, Serban Chiricescu · 21 de julio de 2026

  Enterprises will not deploy AI agents they cannot trust, and the most-cited reason for distrust is hallucination: confident, fluent output that is simply not true. The common response is to wait for a model that does not hallucinate. We argue that this is the wrong target. Large language models are,…

• Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

  Devina Jain, David Hartmann, Chuan Li · 21 de julio de 2026

  LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round att…

• Stress Testing Concept Erasure with Large Language Model Agents

  Yuyang Xue, Feng Chen, Zhihua Liu, Edward Moroshko, Jingyu Sun, Steven McDonagh, Sotirios A. Tsaftaris · 21 de julio de 2026

  Concept erasure aims to remove semantic concepts from a trained generative model and is increasingly important for responsible AI deployment. However, verifying whether a model has robustly removed targeted concepts remains a critical challenge. Existing evaluation methods are typically pre-defined …

• TAPAS: Throughput-adaptive Perception for Autonomous Systems

  Aman Vyas, Vasista Kodumagulla, Zain Taufique, Pasi Liljeberg, Anil Kanduri · 21 de julio de 2026

  Autonomous systems rely on a perception module to navigate through dynamic environments. In real-world scenarios, the perception module's throughput requirements vary at runtime due to changes in scene complexity. However, existing perception strategies assume a fixed FPS and static model-to-cluster…

• TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment

  Changyue Li, Jiaming He, Youliang Yuan, Jialin Wu, Boxi Yu, Zhicong Huang, Pinjia He · 21 de julio de 2026

  Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment. In practice, service providers need to recover models' safety without re-running full alignment, or destroying the utility gained from …

• A Survey on Unlearnable Data

  Jiahao Li, Yiqiang Chen, Yunbing Xing, Yang Gu, Xiangyuan Lan · 21 de julio de 2026

  Unlearnable data (ULD) has emerged as an innovative defense technique to prevent machine learning models from learning meaningful patterns from specific data, thus protecting data privacy and security. By introducing perturbations to the training data, ULD degrades model performance, making it diffi…

• A Multi-Model Hybrid Defense Approach Against White-box Adversarial Attacks in Computer Network Traffic

  Khushnaseeb Roshan · 21 de julio de 2026

  It is crucial to safeguard computer networks from evolving network security threats and unknown cyberattacks. An essential tool for protecting computer networks against unknown cyber threats is Network Intrusion Detection System (NIDS). However, NIDS faces a major security concern due to its suscept…

• Signal-based Model Access Risk Analysis for AI System Operations Security

  Maria Mahbub, Steven Young, Amir Sadovnik, Edmon Begoli, Chris Rugenstein, Donald Coulter, Anthony Ayodele · 21 de julio de 2026

  Artificial intelligence (AI) systems are now ubiquitous across domains such as security, finance, healthcare, consumer technology, and large-scale cloud services, where they process massive volumes of data and make consequential decisions daily. This widespread adoption has created a broad attack su…

• Learning from World Feedback: Why Model Uncertainty Fails as a Risk Signal in Model-Based RL

  Zhaohui Wang · 21 de julio de 2026

  The RLxF programme argues that learning signals should come from world feedback rather than from internal model proxies. We instantiate this position in safe model-based control and distil it into three concrete design principles. Empirically, across four world-model architectures spanning a 2x MSE …

• Integrating High-Level Requirements to Low-Level Tests with Machine-Readable V&V Specifications

  Mansur Arief, Nur Ahmad Khatim, Ali Akarma, Ahmad Alfan Alfian Irfan · 21 de julio de 2026

  Modern software teams have mature tools for low-level testing, such as pytest, JUnit, and Jest, which make it inexpensive to write unit tests and run them on every commit. Systems engineering, in parallel, has developed rigorous principles for design verification and validation (V&V), which has work…

• Reasoning as a Double-Edged Sword: Architecture and Cross-Stage Robustness in Vision-Language-Action Models

  Tuan Duong Trinh, Naveed Akhtar, Basim Azam · 21 de julio de 2026

  Does adding a reasoning step make a Vision-Language-Action (VLA) model more robust to perturbation? Intuitively, a policy that reasons before acting should absorb a perturbed input better than one that maps observations directly to actions. We test this premise head-on across three models that span …

• Lookahead Branching for Neural Network Verification

  Liam Davis, Duo Zhou, Huan Zhang, Guy Katz, Clark Barrett, Haoze Wu · 21 de julio de 2026

  In this work, we investigate the effect of lookahead branching strategies in neural network verification. We present a general recipe to integrate lookahead into any branch-and-bound verifier and demonstrate how one of the current state-of-the-art branching heuristics, FSB, can be viewed as a specia…

• Geometric origin of adversarial vulnerability in deep learning

  Yixiong Ren, Wenkang Du, Jianhui Zhou, Haiping Huang · 21 de julio de 2026

  Balancing training accuracy and adversarial robustness has beeen a challenge since the birth of deep learning. Here, we introduce a geometry-aware deep learning framework that leverages layer-wise local training to sculpt the internal representations of deep neural networks. This framework promotes …

• Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

  Halima Bouzidi, Mboutidem Ekemini Mkpong, Mohammad Abdullah Al Faruque · 20 de julio de 2026

  Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability. We propose Lucid, a black-box adversarial framework that compromises multimodal memory pip…

• Harmonizing AI Safety Thresholds

  Wilber Sean Anterola, Matthew Ball, Luis F. Lafuerza, Markov Grey · 20 de julio de 2026

  Frontier AI companies have published capability thresholds that differ substantially, making it difficult for third parties to verify whether a threshold has been crossed or to compare requirements across companies. Moreover, without common minimum thresholds, risk mitigation may be inconsistent, cr…

• Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents

  Paul Kassianik, Blaine Nelson, Yaron Singer · 20 de julio de 2026

  Security-agent evaluations commonly measure peak offensive capability under generous inference budgets, emphasizing vulnerability discovery, exploit development, penetration testing, and CTF completion. Such measurements are useful but incomplete: in operational security, every reasoning step, tool …

• SpeechGuard: Online Defense against Backdoor Attacks on Speech Recognition Models

  Jinwen Xin, Xixiang Lv · 20 de julio de 2026

  Backdoor attacks pose a critical threat to neural network models, allowing attackers to implant a backdoor during the training phase by manipulating a small portion of the training data. In security-sensitive applications such as voice interaction for autonomous driving, the presence of backdoor att…

• ADS-C: Antidistillation Sampling for Classification

  Khawaja Abaid Ullah, Mohammad Javad Khojasteh · 20 de julio de 2026

  Knowledge distillation enables an adversary to replicate a proprietary classifier by querying its prediction interface and training a surrogate on the returned probability vectors. Antidistillation sampling, proposed for large language models, counters this threat with an input-dependent, gradient-d…