Adversarial Robustness in Machine Learning

Últimos artículos

• Rectifying Adversarial Examples Using Their Vulnerabilities

  Fumiya Morimoto, Ryuto Morita, Satoshi Ono · 5 de enero de 2026

  Deep neural network-based classifiers are prone to errors when processing adversarial examples (AEs). AEs are minimally perturbed input data undetectable to humans posing significant risks to security-dependent applications. Hence, extensive research has been undertaken to develop defense mechanisms…

• The Trojan in the Vocabulary: Stealthy Sabotage of LLM Composition

  Xiaoze Liu, Weichen Yu, Matt Fredrikson, Xiaoqian Wang, Jing Gao · 5 de enero de 2026

  The open-weight LLM ecosystem is increasingly defined by model composition techniques (such as weight merging, speculative decoding, and vocabulary expansion) that remix capabilities from diverse sources. A critical prerequisite for applying these methods across different model families is tokenizer…

• Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm

  Changshun Wu, Weicheng He, Chih-Hong Cheng, Xiaowei Huang, Saddek Bensalem · 5 de enero de 2026

  Out-of-distribution (OoD) inputs pose a persistent challenge to deep learning models, often triggering overconfident predictions on non-target objects. While prior work has primarily focused on refining scoring functions and adjusting test-time thresholds, such algorithmic improvements offer only in…

• AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving

  Shuo Xing, Hongyuan Hua, Xiangbo Gao, Shenzhe Zhu, Renjie Li, Kexin Tian, Xiaopeng Li, Heng Huang, Tianbao Yang, Zhangyang Wang, Yang Zhou, Huaxiu Yao, Zhengzhong Tu · 5 de enero de 2026

  Recent advancements in large vision language models (VLMs) tailored for autonomous driving (AD) have shown strong scene understanding and reasoning capabilities, making them undeniable candidates for end-to-end driving systems. However, limited work exists on studying the trustworthiness of DriveVLM…

• PatchBlock: A Lightweight Defense Against Adversarial Patches for Embedded EdgeAI Devices

  Nandish Chattopadhyay, Abdul Basit, Amira Guesmi, Muhammad Abdullah Hanif, Bassem Ouni, Muhammad Shafique · 5 de enero de 2026

  Adversarial attacks pose a significant challenge to the reliable deployment of machine learning models in EdgeAI applications, such as autonomous driving and surveillance, which rely on resource-constrained devices for real-time inference. Among these, patch-based adversarial attacks, where small ma…

• Cracking IoT Security: Can LLMs Outsmart Static Analysis Tools?

  Jason Quantrill, Noura Khajehnouri, Zihan Guo, Manar H. Alalfi · 5 de enero de 2026

  Smart home IoT platforms such as openHAB rely on Trigger Action Condition (TAC) rules to automate device behavior, but the interplay among these rules can give rise to interaction threats, unintended or unsafe behaviors emerging from implicit dependencies, conflicting triggers, or overlapping condit…

• Adversarial Samples Are Not Created Equal

  Jennifer Crawford, Amol Khanna, Fred Lu, Amy R. Wagoner, Stella Biderman, Andre T. Nguyen, Edward Raff · 5 de enero de 2026

  Over the past decade, numerous theories have been proposed to explain the widespread vulnerability of deep neural networks to adversarial evasion attacks. Among these, the theory of non-robust features proposed by Ilyas et al. has been widely accepted, showing that brittle but predictive features of…

• Large Empirical Case Study: Go-Explore adapted for AI Red Team Testing

  Manish Bhatt, Adrian Wood, Idan Habler, Ammar Al-Kahfah · 5 de enero de 2026

  Production LLM agents with tool-using capabilities require security testing despite their safety training. We adapt Go-Explore to evaluate GPT-4o-mini across 28 experimental runs spanning six research questions. We find that random-seed variance dominates algorithmic parameters, yielding an 8x sprea…

• Mapping Human Anti-collusion Mechanisms to Multi-agent AI

  Jamiu Adekunle Idowu, Ahmed Almasoud, Ayman Alfahid · 5 de enero de 2026

  As multi-agent AI systems become increasingly autonomous, evidence shows they can develop collusive strategies similar to those long observed in human markets and institutions. While human domains have accumulated centuries of anti-collusion mechanisms, it remains unclear how these can be adapted to…

• Audited Skill-Graph Self-Improvement for Agentic LLMs via Verifiable Rewards, Experience Synthesis, and Continual Memory

  Ken Huang, Jerry Huang · 1 de enero de 2026

  Reinforcement learning is increasingly used to transform large language models into agentic systems that act over long horizons, invoke tools, and manage memory under partial observability. While recent work has demonstrated performance gains through tool learning, verifiable rewards, and continual …

• A Comprehensive Study of Deep Learning Model Fixing Approaches

  Hanmo You, Zan Wang, Zishuo Dong, Luanqi Mo, Jianjun Zhao, Junjie Chen · 1 de enero de 2026

  Deep Learning (DL) has been widely adopted in diverse industrial domains, including autonomous driving, intelligent healthcare, and aided programming. Like traditional software, DL systems are also prone to faults, whose malfunctioning may expose users to significant risks. Consequently, numerous ap…

• Prompt-Induced Over-Generation as Denial-of-Service: A Black-Box Attack-Side Benchmark

  Manu, Yi Guo, Jo Plested, Tim Lynar, Kanchana Thilakarathna, Nirhoshan Sivaroopan, Jack Yang, Wangli Yang · 1 de enero de 2026

  Large language models (LLMs) can be driven into over-generation, emitting thousands of tokens before producing an end-of-sequence (EOS) token. This degrades answer quality, inflates latency and cost, and can be weaponized as a denial-of-service (DoS) attack. Recent work has begun to study DoS-style …

• Projection-based Adversarial Attack using Physics-in-the-Loop Optimization for Monocular Depth Estimation

  Takeru Kusakabe, Yudai Hirose, Mashiho Mukaida, Satoshi Ono · 1 de enero de 2026

  Deep neural networks (DNNs) remain vulnerable to adversarial attacks that cause misclassification when specific perturbations are added to input images. This vulnerability also threatens the reliability of DNN-based monocular depth estimation (MDE) models, making robustness enhancement a critical ne…

• Tubular Riemannian Laplace Approximations for Bayesian Neural Networks

  Rodrigo Pereira David · 1 de enero de 2026

  Laplace approximations are among the simplest and most practical methods for approximate Bayesian inference in neural networks, yet their Euclidean formulation struggles with the highly anisotropic, curved loss surfaces and large symmetry groups that characterize modern deep models. Recent work has …

• CPR: Causal Physiological Representation Learning for Robust ECG Analysis under Distribution Shifts

  Shunbo Jia, Caizhi Liao · 1 de enero de 2026

  Deep learning models for Electrocardiogram (ECG) diagnosis have achieved remarkable accuracy but exhibit fragility against adversarial perturbations, particularly Smooth Adversarial Perturbations (SAP) that mimic biological morphology. Existing defenses face a critical dilemma: Adversarial Training …

• Adversarial Lens: Exploiting Attention Layers to Generate Adversarial Examples for Evaluation

  Kaustubh Dhole · 1 de enero de 2026

  Recent advances in mechanistic interpretability suggest that intermediate attention layers encode token-level hypotheses that are iteratively refined toward the final output. In this work, we exploit this property to generate adversarial examples directly from attention-layer token distributions. Un…

• RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

  Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang · 1 de enero de 2026

  Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, …

• Probabilistically Tightened Linear Relaxation-based Perturbation Analysis for Neural Network Verification

  Luca Marzari, Ferdinando Cicalese, Alessandro Farinelli · 1 de enero de 2026

  We present $\textbf{P}$robabilistically $\textbf{T}$ightened $\textbf{Li}$near $\textbf{R}$elaxation-based $\textbf{P}$erturbation $\textbf{A}$nalysis ($\texttt{PT-LiRPA}$), a novel framework that combines over-approximation techniques from LiRPA-based approaches with a sampling-based method to comp…

• HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debate

  Shenzhe Zhu · 1 de enero de 2026

  Large language models (LLMs) are equipped with safety mechanisms to detect and block harmful queries, yet current alignment approaches primarily focus on overtly dangerous content and overlook more subtle threats. However, users can often disguise harmful intent through covert rephrasing that preser…

• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment

  Lijun Zhang, Lin Li, Wei Wei, Yajie Qi, Huizhong Song, Jun Wang, Yaodong Yang, Jiye Liang · 1 de enero de 2026

  When fine-tuning pre-trained Language Models (LMs) to exhibit desired behaviors, maintaining control over risk is critical for ensuring both safety and trustworthiness. Most existing safety alignment methods, such as Safe RLHF and SACPO, typically operate under a risk-neutral paradigm that is insuff…

• DivQAT: Enhancing Robustness of Quantized Convolutional Neural Networks against Model Extraction Attacks

  Kacem Khaled, Felipe Gohring de Magalh\~aes, Gabriela Nicolescu · 1 de enero de 2026

  Convolutional Neural Networks (CNNs) and their quantized counterparts are vulnerable to extraction attacks, posing a significant threat of IP theft. Yet, the robustness of quantized models against these attacks is little studied compared to large models. Previous defenses propose to inject calculate…

• Paired Seed Evaluation: Statistical Reliability for Learning-Based Simulators

  Udit Sharma · 1 de enero de 2026

  Machine learning systems appear stochastic but are deterministically random, as seeded pseudorandom number generators produce identical realisations across executions. Learning-based simulators are widely used to compare algorithms, design choices, and interventions under such dynamics, yet evaluati…

• STED and Consistency Scoring: A Framework for Evaluating LLM Structured Output Reliability

  Guanghui Wang, Jinze Yu, Xing Zhang, Dayuan Jiang, Yin Song, Tomal Deb, Xuefeng Liu, Peiyang He · 1 de enero de 2026

  Large Language Models (LLMs) are increasingly deployed for structured data generation, yet output consistency remains critical for production applications. We introduce a comprehensive framework for evaluating and improving consistency in LLM-generated structured outputs. Our approach combines: (1) …

• The Drill-Down and Fabricate Test (DDFT): A Protocol for Measuring Epistemic Robustness in Language Models

  Rahul Baxi · 1 de enero de 2026

  Current language model evaluations measure what models know under ideal conditions but not how robustly they know it under realistic stress. Static benchmarks like MMLU and TruthfulQA cannot distinguish a model that lacks knowledge from one whose verification mechanisms collapse when information deg…

• Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation

  Deep Shankar Pandey, Hyomin Choi, Qi Yu · 1 de enero de 2026

  Evidential deep learning (EDL) models, based on Subjective Logic, introduce a principled and computationally efficient way to make deterministic neural networks uncertainty-aware. The resulting evidential models can quantify fine-grained uncertainty using learned evidence. However, the Subjective-Lo…