Adversarial Robustness in Machine Learning

Últimos artículos

• Interval Certifications for Multilayered Perceptrons via Lattice Traversal

  Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris, Jo\~ao Marques-Silva · 13 de julio de 2026

  In this work we present a rigorous theoretical framework to a foundational problem of AI safety, namely adversarial robustness. In particular, we show that the adversarial robustness problem can be reduced to a lattice traversal problem. Each element of this lattice corresponds to an interval, i.e.,…

• Statistically Undetectable Backdoors in Deep Neural Networks

  Andrej Bogdanov, Alon Rosen, Neekon Vafa · 13 de julio de 2026

  We show how an adversarial model trainer can plant backdoors in a large class of deep, feedforward neural networks. These backdoors are statistically undetectable in the white-box setting, meaning that the backdoored and honestly trained models are close in total variation distance, even given the f…

• Optimizing Against Safety Representations: Activation-Guided Adversarial Suffixes and the Geometry of Refusal

  Ege \c{C}akar, Hannah Guan, Kayden Kehe · 13 de julio de 2026

  Behavioral alignment in large language models often masks fragile internal safety representations. Recent work suggests that refusal behavior is mediated by low-dimensional directions in activation space. This raises questions about how such representations are structured, localized, and accessed by…

• Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitors

  Maheep Chaudhary, Fazl Barez · 13 de julio de 2026

  White-box monitoring is increasingly adopted as an auditing tool as Large Language Models (LLMs) are deployed in daily operations to ensure safe model behavior. However, white-box monitors can be circumvented, and the mechanisms underlying such evasion have not been systematically characterized, nor…

• Secure-by-Disguise: A Systematic Evaluation of Image Disguising for Confidential Medical Image Modeling

  Jason Rojas, Jiajie He, Yash Patel, Yuechun Gu, Zeyun Yu, Keke Chen · 13 de julio de 2026

  Cloud-based deep learning enables large-scale medical image analysis but raises significant privacy concerns when sensitive patient images are outsourced for model development. Image disguising has recently emerged as a promising privacy-enhancing technology (PET) that transforms images into visuall…

• Robustifying Vision-Language Models via Test-Time Prompt Adaptation

  Xingyu Zhu, Huanshen Wu, Shuo Wang, Beier Zhu, Jiannan Ge, Jiaheng Zhang, Long Chen · 13 de julio de 2026

  Pre-trained Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under adversarial perturbations. Existing test-time adaptation methods typically rely on sample-level confidence heuristics, overlooking the intrinsic distributional…

• Neuro-Agentic Control: A Deep Learning-based LLM-Powered Agentic AI Framework for Controlling Security Controls

  Saroj Gopali, Bipin Chhetri, Deepika Giri, Sima Siami-Namini, Akbar Siami Namin · 13 de julio de 2026

  Cyberattacks on operational technology are increasingly causing costly downtime and physical damage, exposing the limitations of traditional rule-based monitoring in industrial IoT environments. While Large Language Models (LLMs) have strong semantic reasoning abilities to assist in decision support…

• Prediction-Powered Active Testing

  Kianoosh Ashouritaklimi, Valentin Kilian, Daolang Huang, Tom Rainforth, Fran\c{c}ois Caron · 10 de julio de 2026

  Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled. However, existing estimators fail to exploit the informative predictions of powerful black--box models, even though such predictions are increasingly available in sett…

• Efficient Safety Alignment of Language Models via Latent Personality Traits

  Mohamed Amine Merzouk, Nolan Smyth, Damiano Fornasiere, Linh Le, David Williams-King, Adam Oberman · 10 de julio de 2026

  Current safety methods for large language models are known to be vulnerable to adversarial attacks, motivating research into robust alternatives. Latent Adversarial Training (LAT) is among the most effective defenses, but can degrade utility and requires training on large datasets of harmful prompts…

• Modular Pretraining Enables Access Control

  Ethan Roland, Murat Cubuktepe, Erick Martinez, Stijn Servaes, Keenan Pepper, Mike Vaiana, Diogo Schwerz de Lucena, Judd Rosenblatt, Addie Foote, Cem Anil, Alex Cloud · 10 de julio de 2026

  AI developers face a dual-use dilemma. An AI capability that helps one user cure a disease can help another synthesize one. This dilemma could be resolved with access control, limiting dual-use AI capabilities to trusted deployments with a legitimate need. A gold standard for access control would be…

• Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning

  Zifan Zhang, Minghong Fang, Dianwei Chen, Zhuqing Liu, Prashant Khanduri, Xianfeng Yang, Anupam Das, Yuchen Liu · 10 de julio de 2026

  Federated reinforcement learning (FRL) is crucial for enabling collaborative learning across multiple agents without sharing raw data, thereby enhancing privacy and scalability in the decision-making process within dynamic vehicular environments. However, poisoning attacks pose a significant threat …

• Robust Bayesian Decision Making under Adversarial Uncertainty

  Haripriya Harikumar, Sammie Katt, Yasir Zubayr Barlas, Samuel Kaski · 10 de julio de 2026

  Scientific experiments are often designed to maximize information gain, yet in many applications the primary objective is to support reliable downstream decision-making. Existing decision-aware experimental design and active learning methods typically assume well-specified outcome models and implici…

• LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

  Arthur Chiron (IRIT, EPE UT), Franck Mamalet (IRIT, DTIPG - SNCF, UT3), Thomas Massena (IRIT, DTIPG - SNCF, UT3), Thomas Deltort (IRIT), Mathieu Serrurier (IRIT, UT2J) · 10 de julio de 2026

  While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge. Lipschitz-constrained models guarantee robustness by design, yet the manual selection of…

• Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator

  Shiping Yang, Shining Liang, Weihao Liu, Wenbiao Ding, Linjun Shou, Lu Cheng, Angel X. Chang · 10 de julio de 2026

  Identifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator …

• Persuasion Attacks Can Decrease Effectiveness of CoT Monitoring

  Jennifer Za, Julija Bainiaksina, Nikita Ostrovsky, Tanush Chopra, Victoria Krakovna · 10 de julio de 2026

  Chain-of-thought (CoT) monitoring is a promising safety mechanism for AI agents, based on the premise that visible reasoning traces can surface misaligned or deceptive behavior. While effective in standard scenarios, recent work highlights that LLMs remain vulnerable to persuasion-based jailbreaks, …

• LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection

  Vincent L\'eb\'e (IRIT, DTIPG - SNCF, UT3), Yannick Prudent (IRIT, DTIPG - SNCF, UT3), Corentin Friedrich (IRIT, DTIPG - SNCF, UT3), Thomas Massena (IRIT, DTIPG - SNCF, UT3), Ronan Sicre (IRIT), Franck Mamalet · 9 de julio de 2026

  Object detectors have many applications in safety-critical systems, but they are known to be sensitive to worst-case perturbations such as adversarial attacks, which limits their applicability in real-world scenarios. Compared with classification, adversarial robustness for object detection has rece…

• Safety Degradation in AI Agents

  Cheng Yu, Benedikt Stroebl, Diyi Yang, Orestis Papakyriakopoulos · 9 de julio de 2026

  Despite the growing integration of retrieval-enabled AI agents into society, their safety and ethical behavior remain inadequately understood. In particular, the integration of LLMs and AI agents with external information sources and real-world environments raises critical questions about how they e…

• InferNet: Exploiting Aggregate GPU Profiles as Side-Channel for DNN Architecture Inference

  Raja Hasnain Anwar, Jonah O'Brien Weiss, Tiago Alves, Sandip Kundu · 9 de julio de 2026

  Deep Neural Networks (DNNs) have become ubiquitous for their ability to solve problems across various domains, including computer vision, natural language processing, and speech recognition. However, as their adoption grows, they face a range of security threats, such as model stealing, architecture…

• Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations

  Silvia Santano · 9 de julio de 2026

  Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output. Detecting unfaithfulness, though, requires controlled experimental interventions, which cannot be applied to evaluation transcrip…

• Predicting LLM Safety Before Release by Simulating Deployment

  Marcus Williams, Hannah Sheahan, Cameron Raymond, Tomek Korbak, Deng Pan, Peilin Yang, Leon Maksin, Ningyi Xie, Phillip Guo, Ian Kivlichan, Micah Carroll · 9 de julio de 2026

  Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recogn…

• Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees

  Yilei Chen, Vittorio Giammarino, James Queeney, Ioannis Ch. Paschalidis · 9 de julio de 2026

  Adversarial Imitation Learning (AIL) faces challenges with sample inefficiency because of its reliance on sufficient on-policy data to evaluate the performance of the current policy during reward function updates. In this work, we study the convergence properties and sample complexity of off-policy …

• Prior-aware and Context-guided Group Sampling for Active Probabilistic Subsampling

  Beomgu Kang, Hyunseok Seo · 9 de julio de 2026

  Subsampling significantly reduces the number of measurements, thereby streamlining data processing and transfer overhead, and shortening acquisition time across diverse real-world applications. The recently introduced Active Deep Probabilistic Subsampling (A-DPS) approach jointly optimizes both the …

• ORAN-DEFEND: Subspace Detection and Sanitization of Backdoor DRL xApps in Open RAN

  Md Raihan Uddin, Fatemeh Lotfi, Tolunay Seyfi, Fatemeh Afghah · 9 de julio de 2026

  Open Radio Access Networks (O-RAN) increasingly delegate near-real-time control to deep reinforcement learning (DRL) xApps obtained from third-party vendors, creating a new supply-chain attack surface. A backdoor policy behaves optimally until an adversary injects a covert trigger into the observed …

• Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

  Mingguang Chen, Licheng Wang, Bo Qu · 9 de julio de 2026

  AI systems increasingly participate in their own improvement: revising their outputs, adapting their own harnesses during deployment, training on data they generate, and, increasingly, conducting AI research itself. This literature is described under a vocabulary ("self-refine," "self-reward," "self…

• POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

  Zhangheng LI, Jianing Zhu, Junyuan Hong, Sungmin Eum, Shuowen Hu, Suya You, Zhangyang Wang · 9 de julio de 2026

  Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and visual data, where privacy-sensitive examples could be unintentionally encoded, raising concerns about privacy or copyright violation. To this end, Mu…