Adversarial Robustness in Machine Learning

Últimos artículos

• RECAP: A Resource-Efficient Method for Adversarial Prompting in Large Language Models

  Rishit Chugh · 23 de enero de 2026

  The deployment of large language models (LLMs) has raised security concerns due to their susceptibility to producing harmful or policy-violating outputs when exposed to adversarial prompts. While alignment and guardrails mitigate common misuse, they remain vulnerable to automated jailbreaking method…

• QUAIL: Quantization Aware Unlearning for Mitigating Misinformation in LLMs

  Himanshu Mishra, Kanwal Mehreen · 23 de enero de 2026

  Machine unlearning aims to remove specific knowledge (e.g., copyrighted or private data) from a trained model without full retraining. In practice, models are often quantized (e.g., 4-bit) for deployment, but we find that quantization can catastrophically restore forgotten information [1]. In this p…

• Attributing and Exploiting Safety Vectors through Global Optimization in Large Language Models

  Fengheng Chu, Jiahao Chen, Yuhong Wang, Jun Wang, Zhihui Fu, Shouling Ji, Songze Li · 23 de enero de 2026

  While Large Language Models (LLMs) are aligned to mitigate risks, their safety guardrails remain fragile against jailbreak attacks. This reveals limited understanding of components governing safety. Existing methods rely on local, greedy attribution that assumes independent component contributions. …

• On damage of interpolation to adversarial robustness in regression

  Jingfu Peng, Yuhong Yang · 23 de enero de 2026

  Deep neural networks (DNNs) typically involve a large number of parameters and are trained to achieve zero or near-zero training error. Despite such interpolation, they often exhibit strong generalization performance on unseen data, a phenomenon that has motivated extensive theoretical investigation…

• Beyond Predictive Uncertainty: Reliable Representation Learning with Structural Constraints

  Yiyao Yang · 23 de enero de 2026

  Uncertainty estimation in machine learning has traditionally focused on the prediction stage, aiming to quantify confidence in model outputs while treating learned representations as deterministic and reliable by default. In this work, we challenge this implicit assumption and argue that reliability…

• Provable Robustness in Multimodal Large Language Models via Feature Space Smoothing

  Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang · 23 de enero de 2026

  Multimodal large language models (MLLMs) exhibit strong capabilities across diverse applications, yet remain vulnerable to adversarial perturbations that distort their feature representations and induce erroneous predictions. To address this vulnerability, we propose the Feature-space Smoothing (FS)…

• Turn-Based Structural Triggers: Prompt-Free Backdoors in Multi-Turn LLMs

  Yiyang Lu, Jinwen He, Yue Zhao, Kai Chen, Ruigang Liang · 22 de enero de 2026

  Large Language Models (LLMs) are widely integrated into interactive systems such as dialogue agents and task-oriented assistants. This growing ecosystem also raises supply-chain risks, where adversaries can distribute poisoned models that degrade downstream reliability and user trust. Existing backd…

• How Worst-Case Are Adversarial Attacks? Linking Adversarial and Statistical Robustness

  Giulio Rossolini · 22 de enero de 2026

  Adversarial attacks are widely used to evaluate model robustness, yet their validity as proxies for robustness to random perturbations remains debated. We ask whether an adversarial perturbation provides a representative estimate of robustness under random noise of the same magnitude, or instead ref…

• Unraveling LLM Jailbreaks Through Safety Knowledge Neurons

  Chongwen Zhao, Yutong Ke, Kaizhu Huang · 22 de enero de 2026

  Large Language Models (LLMs) are increasingly attracting attention in various applications. Nonetheless, there is a growing concern as some users attempt to exploit these models for malicious purposes, including the synthesis of controlled substances and the propagation of disinformation, a techniqu…

• Large Language Models Encode Semantics and Alignment in Linearly Separable Representations

  Baturay Saglam, Paul Kassianik, Blaine Nelson, Sajana Weerawardhena, Yaron Singer, Amin Karbasi · 22 de enero de 2026

  Understanding the latent space geometry of large language models (LLMs) is key to interpreting their behavior and improving alignment. Yet it remains unclear to what extent LLMs linearly organize representations related to semantic understanding. To explore this, we conduct a large-scale empirical s…

• Gradient Structure Estimation under Label-Only Oracles via Spectral Sensitivity

  Jun Liu, Leo Yu Zhang, Fengpeng Li, Isao Echizen, Jiantao Zhou · 22 de enero de 2026

  Hard-label black-box settings, where only top-1 predicted labels are observable, pose a fundamentally constrained yet practically important feedback model for understanding model behavior. A central challenge in this regime is whether meaningful gradient information can be recovered from such discre…

• Fine-Grained Traceability for Transparent ML Pipelines

  Liping Chen, Mujie Liu, Haytham Fayek · 22 de enero de 2026

  Modern machine learning systems are increasingly realised as multistage pipelines, yet existing transparency mechanisms typically operate at a model level: they describe what a system is and why it behaves as it does, but not how individual data samples are operationally recorded, tracked, and verif…

• GCG Attack On A Diffusion LLM

  Ruben Neyroud, Sam Corley · 22 de enero de 2026

  While most LLMs are autoregressive, diffusion-based LLMs have recently emerged as an alternative method for generation. Greedy Coordinate Gradient (GCG) attacks have proven effective against autoregressive models, but their applicability to diffusion language models remains largely unexplored. In th…

• INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systems

  Yijin Zhou, Xiaoya Lu, Dongrui Liu, Junchi Yan, Jing Shao · 22 de enero de 2026

  The rapid advancement of Large Language Model (LLM)-based Multi-Agent Systems (MAS) has introduced significant security vulnerabilities, where malicious influence can propagate virally through inter-agent communication. Conventional safeguards often rely on a binary paradigm that strictly distinguis…

• Learning from Discriminatory Training Data

  Przemyslaw A. Grabowicz, Nicholas Perello, Kenta Takatsu · 22 de enero de 2026

  Supervised learning systems are trained using historical data and, if the data was tainted by discrimination, they may unintentionally learn to discriminate against protected groups. We propose that fair learning methods, despite training on potentially discriminatory datasets, shall perform well on…

• LLM Security and Safety: Insights from Homotopy-Inspired Prompt Obfuscation

  Luis Lazo, Hamed Jelodar, Roozbeh Razavi-Far · 22 de enero de 2026

  In this study, we propose a homotopy-inspired prompt obfuscation framework to enhance understanding of security and safety vulnerabilities in Large Language Models (LLMs). By systematically applying carefully engineered prompts, we demonstrate how latent model behaviors can be influenced in unexpect…

• CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models

  Nay Myat Min, Long H. Pham, Hongyu Zhang, Jun Sun · 22 de enero de 2026

  Single-pass hallucination detectors rely on internal telemetry (e.g., uncertainty, hidden-state geometry, and attention) of large language models, implicitly assuming hallucinations leave separable traces in these signals. We study a white-box, model-side adversary that fine-tunes lightweight LoRA a…

• SilentDrift: Exploiting Action Chunking for Stealthy Backdoor Attacks on Vision-Language-Action Models

  Bingxin Xu, Yuzhang Shang, Binghui Wang, Emilio Ferrara · 22 de enero de 2026

  Vision-Language-Action (VLA) models are increasingly deployed in safety-critical robotic applications, yet their security vulnerabilities remain underexplored. We identify a fundamental security flaw in modern VLA systems: the combination of action chunking and delta pose representations creates an …

• DDSA: Dual-Domain Strategic Attack for Spatial-Temporal Efficiency in Adversarial Robustness Testing

  Jinwei Hu, Shiyuan Meng, Yi Dong, Xiaowei Huang · 22 de enero de 2026

  Image transmission and processing systems in resource-critical applications face significant challenges from adversarial perturbations that compromise mission-specific object classification. Current robustness testing methods require excessive computational resources through exhaustive frame-by-fram…

• VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models

  Yongchao Huang · 22 de enero de 2026

  Joint Embedding Predictive Architectures (JEPA) offer a scalable paradigm for self-supervised learning by predicting latent representations rather than reconstructing high-entropy observations. However, existing formulations rely on \textit{deterministic} regression objectives, which mask probabilis…

• HyperNet-Adaptation for Diffusion-Based Test Case Generation

  Oliver Wei{\ss}l, Vincenzo Riccio, Severin Kacianka, Andrea Stocco · 22 de enero de 2026

  The increasing deployment of deep learning systems requires systematic evaluation of their reliability in real-world scenarios. Traditional gradient-based adversarial attacks introduce small perturbations that rarely correspond to realistic failures and mainly assess robustness rather than functiona…

• SpooFL: Spoofing Federated Learning

  Isaac Baglin, Xiatian Zhu, Simon Hadfield · 22 de enero de 2026

  Traditional defenses against Deep Leakage (DL) attacks in Federated Learning (FL) primarily focus on obfuscation, introducing noise, transformations or encryption to degrade an attacker's ability to reconstruct private data. While effective to some extent, these methods often still leak high-level i…

• Auditing Language Model Unlearning via Information Decomposition

  Anmol Goel, Alan Ritter, Iryna Gurevych · 22 de enero de 2026

  We expose a critical limitation in current approaches to machine unlearning in language models: despite the apparent success of unlearning algorithms, information about the forgotten data remains linearly decodable from internal representations. To systematically assess this discrepancy, we introduc…

• Manipulating Feature Visualizations with Gradient Slingshots

  Dilyara Bareeva, Marina M. -C. H\"ohne, Alexander Warnecke, Lukas Pirch, Klaus-Robert M\"uller, Konrad Rieck, Sebastian Lapuschkin, Kirill Bykov · 21 de enero de 2026

  Feature Visualization (FV) is a widely used technique for interpreting concepts learned by Deep Neural Networks (DNNs), which synthesizes input patterns that maximally activate a given feature. Despite its popularity, the trustworthiness of FV explanations has received limited attention. We introduc…

• Kernel-Based Learning of Safety Barriers

  Oliver Sch\"on, Zhengang Zhong, Sadegh Soudjani · 21 de enero de 2026

  The rapid integration of AI algorithms in safety-critical applications such as autonomous driving and healthcare is raising significant concerns about the ability to meet stringent safety standards. Traditional tools for formal safety verification struggle with the black-box nature of AI-driven syst…