Adversarial Robustness in Machine Learning

Últimos artículos

• Random Parameter Noise Does Not Make Exact ReLU Verification Easy

  Mojtaba Soltanalian · 17 de julio de 2026

  We study exact verification of ReLU networks in an adversarial smoothed model. Every network weight and bias is independently perturbed by Gaussian noise, clipped to $[-2,2]$, and rounded to the exact dyadic grid determined by the input bit complexity. We show that, under the standard assumption $\m…

• Decision Making Needs Uncertainty Quantification [Lecture Notes]

  Osvaldo Simeone · 17 de julio de 2026

  Many signal processing systems ultimately exist to {act}. Whenever the state variable that determines the action to be taken by a decision maker, or agent, is uncertain, the way that uncertainty is represented decides how well the agent performs and how much its performance can be trusted. This lect…

• CASP: Learning-Augmented Offline Approximation with Verifiable Certificates and Bounded-Loss PAC Guarantees

  Haifeng Li, Mo Hai · 17 de julio de 2026

  Machine-learned predictions can speed up offline NP-hard optimization, but asking a predictor what to do amounts to asking it to solve the problem, and committing an unchecked prediction forfeits every worst-case guarantee. CASP (Certificate-Augmented Solution Pruning) instead asks which parts of th…

• BadWAM: When World-Action Models Dream Right but Act Wrong

  Qi Li, Xingyi Yang, Xinchao Wang · 17 de julio de 2026

  World-action models (WAMs) are emerging as a promising foundation for embodied control: rather than predicting actions alone, they learn representations that couple action generation with future world prediction. This coupling is often viewed as a source of robustness, interpretability, and safety, …

• Subjective Risk Decomposition: A New View for Uncertainty Quantification

  Raghad Alamri, Michele Caprio, Gavin Brown · 17 de julio de 2026

  We present a novel viewpoint for uncertainty quantification. Uncertainty measures are not primitives, in need of axioms and argumentation, but instead consequences, of higher-level modelling decisions. We show how epistemic and aleatoric uncertainty measures can be derived via decomposition of a sub…

• Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification

  Patricia Medina, Hy P. G. Lam · 17 de julio de 2026

  We study Dynamical System Autoencoders (DSAE) for LiDAR point-cloud classification using spatial coordinates and Product Coefficient feature augmentations. The experiments compare separately trained DSAE architectures at encoder depths $K=1,\ldots,5$ and evaluate the resulting hidden representations…

• Random Logit Scaling: Defending Deep Neural Networks Against Black-Box Score-Based Adversarial Example Attacks

  Hamid Dashtbani, Mehdi Dousti Gandomani, AmirMahdi Sadeghzadeh · 17 de julio de 2026

  Machine learning models are increasingly adapted in various domains. However, adversarial examples pose a significant threat to the reliable deployment of these models. In recent years, some powerful adversarial example attacks have been proposed for the fast and query-efficient generation of advers…

• RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences

  Logan Mondal Bhamidipaty, Mykel Kochenderfer, Subramanian Ramamoorthy · 17 de julio de 2026

  World models are widely used in offline reinforcement learning (RL) to improve sample efficiency and generate experience beyond a fixed dataset. However, they are vulnerable to model exploitation where data coverage is thin. Prior work addresses this either by collecting more expert demonstrations, …

• GeoDetect: Geometric Adversarial Detection for VLPs

  Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie, James Bailey, Sarah Erfani · 17 de julio de 2026

  Vision-language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to adversarial attacks. Although adversarial detection methods have demonstrated success in single-modality settings (either vision or language), their effectiveness and reliability …

• SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification

  SingGuard Team · 16 de julio de 2026

  We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive information extraction, malicious code requests, dangerous tool misuse, and resource exhaustion. We first introduce the NSFA taxonomy, which organizes 185 risk…

• Adversarial Prompting Framework for AI Safety Assessment

  Yash Bhatnagar, Kunal Banerjee, Anirban Chatterjee · 16 de julio de 2026

  Artificial Intelligence (AI), especially Generative AI (GenAI), adoption has increased in industries significantly in recent years. However, the use of these models may also expose systems to new forms of cyberattacks by different malicious actors -- adversarial prompt attack (APA) being one of the …

• Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

  Mohammad Allahbakhsh, Mohammad Hassan Bahari, Moslem Attar-Raouf · 16 de julio de 2026

  Penetration testing traditionally evaluates whether adversaries can exploit weaknesses in software, infrastructure, configurations, or operational controls to achieve security-relevant compromise. This paradigm remains necessary for AI-enabled systems, but it is no longer sufficient. In such systems…

• UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors

  Dima Galat, Marian-Andrei Rizoiu · 16 de julio de 2026

  We investigate which language model evasion attacks survive state-of-the-art adversarial fine-tuning, developing strategies that sweep the top 5 positions on the ELOQUENT 2026 Voight-Kampff leaderboard. While adversarial fine-tuning trivially closes the 2025 winning evasion recipes, we uncover a fun…

• WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

  Z Sun, Q Jiang, S Sheng, L Xiang · 16 de julio de 2026

  Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques. However, these techniques have rarely been adopted in practice mainly for two reasons: i) severely degraded model pe…

• When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

  Na Li, Boyu Kuang, Hongsheng Hu, Liquan Chen, Hyoungshick Kim, Yansong Gao, Anmin Fu · 16 de julio de 2026

  To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT). While substituting synthetic data…

• Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization

  Yuxin Huang, Ziming Hong, Mingming Gong, Wanyu Wang, Jing Zhang, Tongliang Liu · 16 de julio de 2026

  Recent diffusion-based video generation models have enabled high-quality personalized video customization through both tuning-based pipelines, which fine-tune a video diffusion model, and reference-based pipelines such as image-to-video generation. However, these capabilities raise serious concerns …

• Securing LLMs in the Wild: Privacy and Security Challenges at the Edge

  Ren-Yi Huang, Mingchen Li, Dumindu Samaraweera, Morris Chang · 16 de julio de 2026

  Large Language Models (LLMs) are rapidly moving from research settings into the wild, deployed on enterprise infrastructure, personal devices, and edge platforms. While cloud deployments offer scalable compute, concerns over data sovereignty, compliance, latency, and third-party dependence are drivi…

• Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors

  Michael O. Eniolade · 16 de julio de 2026

  Clinical AI models can expose patients to harm when adversarial vulnerabilities go undetected, yet formal security auditing requires statistical expertise, specialized tools, and significant time. We present an open evaluation task, built on METR Task Standard v0.3.0, that tests whether frontier AI …

• GFlowRL: Scaling Distribution-Matching RL to Large Language Models

  Xiaodong Liu, Michael Xu, Jack W. Stokes, Paul Smolensky, Doug Burger, Jianfeng Gao · 16 de julio de 2026

  Generative Flow Networks (GFlowNets) offer a promising alternative to reward-maximizing reinforcement learning (RL) for large reasoning models, encouraging diverse reasoning paths by matching reward distributions rather than collapsing to dominant modes. Recent work shows promise on math and code, b…

• Exact and Certified Data Shapley for Weighted k-Nearest-Neighbor Regression and Soft-Label Prediction

  Zongye Lyu · 15 de julio de 2026

  Data Shapley is the standard principled answer to which training points are worth what, and its k-nearest-neighbor (KNN) specialization is the version deployed in practice: the exact estimator shipped by toolkits such as pyDVL and OpenDataVal. Exact algorithms are known for unweighted KNN and for we…

• Reducing information dependency does not cause training data privacy. Adversarially non-robust features do

  Rasmus Torp, Shailen K. Smith, Adam Breuer · 15 de julio de 2026

  In this paper, we challenge the prevailing view that information dependency (including rote memorization) drives training data exposure to image reconstruction attacks. We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial …

• Adversarial Attacks on Online Handwriting using Salience-based Temporal Editing

  Yataro Tamura, Brian Kenji Iwana, Jiseok Lee · 15 de julio de 2026

  Deep learning models for online handwriting recognition have been shown effective and are increasingly deployed in practical applications. However, their vulnerability to adversarial attacks is still a challenge. Existing adversarial methods are predominantly designed for image-based inputs and typi…

• RAFP: Identifying LLM Lineages via Rare-Region Fingerprints

  Yun-Yun Tsai, Jia Hao Liang, Chuan Guo, Junfeng Yang, Laurens van der Maaten · 15 de julio de 2026

  Large language models (LLMs) are increasingly released under restricted licenses, creating a growing need for robust model ownership verification. Existing fingerprinting methods are often fragile under downstream finetuning, require invasive training modifications, or fail in black-box settings. We…

• Sparse Autoencoders for Interpretable Out-of-Distribution Detection

  Ayush Karmacharya (Purdue University), Luke Luschwitz (Purdue University), Lucia Romero (Purdue University), Yanan Niu (EPFL), Joseph Campbell (Purdue University) · 15 de julio de 2026

  Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection…

• A Symbolic Neural CPU for Quantization-Simulated Writeback and Interpretable Program Execution

  Jose Luis Lima de Jesus Silva · 14 de julio de 2026

  Neural networks can learn algorithmic input-output mappings, but trusting a learned executor requires more than a correct final answer because the state transitions that produce it are usually hidden. To make those transitions visible, we introduce a trace-supervised symbolic neural CPU, a factorize…