Adversarial Robustness in Machine Learning

Últimos artículos

• Detoxifying LLMs via Representation Erasure-Based Preference Optimization

  Nazanin Mohammadi Sepahvand, Eleni Triantafillou, Hugo Larochelle, Doina Precup, Daniel M. Roy, Gintare Karolina Dziugaite · 2 de marzo de 2026

  Large language models (LLMs) trained on webscale data can produce toxic outputs, raising concerns for safe deployment. Prior defenses, based on applications of DPO, NPO, and similar algorithms, reduce the likelihood of harmful continuations, but not robustly so: they are vulnerable to adversarial pr…

• A Variational Estimator for $L_p$ Calibration Errors

  Eug\`ene Berta, Sacha Braun, David Holzm\"uller, Francis Bach, Michael I. Jordan · 2 de marzo de 2026

  Calibration$\unicode{x2014}$the problem of ensuring that predicted probabilities align with observed class frequencies$\unicode{x2014}$is a basic desideratum for reliable prediction with machine learning systems. Calibration error is traditionally assessed via a divergence function, using the expect…

• Partition Function Estimation under Bounded f-Divergence

  Adam Block, Abhishek Shetty · 2 de marzo de 2026

  We study the statistical complexity of estimating partition functions given sample access to a proposal distribution and an unnormalized density ratio for a target distribution. While partition function estimation is a classical problem, existing guarantees typically rely on structural assumptions a…

• Concept-based Adversarial Attack: a Probabilistic Perspective

  Andi Zhang, Xuan Ding, Steven McDonagh, Samuel Kaski · 2 de marzo de 2026

  We propose a concept-based adversarial attack framework that extends beyond single-image perturbations by adopting a probabilistic perspective. Rather than modifying a single image, our method operates on an entire concept - represented by a distribution - to generate diverse adversarial examples. P…

• Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking

  Zhicheng Fang, Jingjie Zheng, Chenxu Fu, Wei Xu · 2 de marzo de 2026

  Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols. We introduce JAILBREAK FOUNDRY (JBF), a system that addresses this gap via a multi…

• To Deceive is to Teach? Forging Perceptual Robustness via Adversarial Reinforcement Learning

  Yicheng Bao, Xuhong Wang, Xin Tan · 27 de febrero de 2026

  Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) exhibit perceptual fragility when confronted with visually complex scenes. This weakness stems from a reliance on finite training datasets, which are prohibitively expensive to scale and impose a ceiling on model robustn…

• CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety

  Umid Suleymanov, Rufiz Bayramov, Suad Gafarli, Seljan Musayeva, Taghi Mammadov, Aynur Akhundlu, Murat Kantarcioglu · 27 de febrero de 2026

  Current safety mechanisms for Large Language Models (LLMs) rely heavily on static, fine-tuned classifiers that suffer from adaptation rigidity, the inability to enforce new governance rules without expensive retraining. To address this, we introduce CourtGuard, a retrieval-augmented multi-agent fram…

• Manifold of Failure: Behavioral Attraction Basins in Language Models

  Sarthak Munshi, Manish Bhatt, Vineeth Sai Narajala, Idan Habler, AmmarnAl-Kahfah, Ken Huang, Blake Gatto · 27 de febrero de 2026

  While prior work has focused on projecting adversarial examples back onto the manifold of natural data to restore safety, we argue that a comprehensive understanding of AI safety requires characterizing the unsafe regions themselves. This paper introduces a framework for systematically mapping the M…

• Multilingual Safety Alignment Via Sparse Weight Editing

  Jiaming Liang, Zhaoxin Wang, Handing Wang · 27 de febrero de 2026

  Large Language Models (LLMs) exhibit significant safety disparities across languages, with low-resource languages (LRLs) often bypassing safety guardrails established for high-resource languages (HRLs) like English. Existing solutions, such as multilingual supervised fine-tuning (SFT) or Reinforceme…

• Conformalized Neural Networks for Federated Uncertainty Quantification under Dual Heterogeneity

  Quang-Huy Nguyen, Jiaqi Wang, Wei-Shinn Ku · 27 de febrero de 2026

  Federated learning (FL) faces challenges in uncertainty quantification (UQ). Without reliable UQ, FL systems risk deploying overconfident models at under-resourced agents, leading to silent local failures despite seemingly satisfactory global performance. Existing federated UQ approaches often addre…

• Poisoned Acoustics

  Harrison Dahme · 27 de febrero de 2026

  Training-data poisoning attacks can induce targeted, undetectable failure in deep neural networks by corrupting a vanishingly small fraction of training labels. We demonstrate this on acoustic vehicle classification using the MELAUDIS urban intersection dataset (approx. 9,600 audio clips, 6 classes)…

• TorchLean: Formalizing Neural Networks in Lean

  Robert Joseph George, Jennifer Cruden, Xiangru Zhong, Huan Zhang, Anima Anandkumar · 27 de febrero de 2026

  Neural networks are increasingly deployed in safety- and mission-critical pipelines, yet many verification and analysis results are produced outside the programming environment that defines and runs the model. This separation creates a semantic gap between the executed network and the analyzed artif…

• Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Study

  Kaizheng Wang, Yunjia Wang, Fabio Cuzzolin, David Moens, Hans Hallez, Siu Lun Chau · 27 de febrero de 2026

  Epistemic uncertainty in neural networks is commonly modeled using two second-order paradigms: distribution-based representations, which rely on posterior parameter distributions, and set-based representations based on credal sets (convex sets of probability distributions). These frameworks are ofte…

• Beyond Dominant Patches: Spatial Credit Redistribution For Grounded Vision-Language Models

  Niamul Hassan Samin, Md Arifur Rahman, Abdullah Ibne Hanif, Juena Ahmed Noshin, Md Ashikur Rahman · 27 de febrero de 2026

  Vision-language models (VLMs) frequently hallucinate objects absent from the input image. We trace this failure to spatial credit collapse: activation credit concentrating on sparse visual patches in early transformer layers, which suppresses contextual evidence and increases reliance on language pr…

• Knob: A Physics-Inspired Gating Interface for Interpretable and Controllable Neural Dynamics

  Siyu Jiang, Sanshuai Cui, Hui Zeng · 27 de febrero de 2026

  Existing neural network calibration methods often treat calibration as a static, post-hoc optimization task. However, this neglects the dynamic and temporal nature of real-world inference. Moreover, existing methods do not provide an intuitive interface enabling human operators to dynamically adjust…

• Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search

  Xun Huang, Simeng Qin, Xiaoshuang Jia, Ranjie Duan, Huanqian Yan, Zhitao Zeng, Fei Yang, Yang Liu, Xiaojun Jia · 27 de febrero de 2026

  As Large Language Models (LLMs) are increasingly used, their security risks have drawn increasing attention. Existing research reveals that LLMs are highly susceptible to jailbreak attacks, with effectiveness varying across language contexts. This paper investigates the role of classical Chinese in …

• Analysis of LLMs Against Prompt Injection and Jailbreak Attacks

  Piyush Jaiswal, Aaditya Pratap, Shreyansh Saraswati, Harsh Kasyap, Somanath Tripathy · 27 de febrero de 2026

  Large Language Models (LLMs) are widely deployed in real-world systems. Given their broader applicability, prompt engineering has become an efficient tool for resource-scarce organizations to adopt LLMs for their own purposes. At the same time, LLMs are vulnerable to prompt-based attacks. Thus, anal…

• Invariant Transformation and Resampling based Epistemic-Uncertainty Reduction

  Sha Hu · 27 de febrero de 2026

  An artificial intelligence (AI) model can be viewed as a function that maps inputs to outputs in high-dimensional spaces. Once designed and well trained, the AI model is applied for inference. However, even optimized AI models can produce inference errors due to aleatoric and epistemic uncertainties…

• Bound to Disagree: Generalization Bounds via Certifiable Surrogates

  Mathieu Bazinet, Valentina Zantedeschi, Pascal Germain · 27 de febrero de 2026

  Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes. In this paper, we tackle these issues by providing new disagreement-based certificates for the gap between the true risk of any two predictors. We then bound the true risk of…

• Training Agents to Self-Report Misbehavior

  Bruce W. Lee, Chen Yueh-Han, Tomek Korbak · 27 de febrero de 2026

  Frontier AI agents may pursue hidden goals while concealing their pursuit from oversight. Alignment training aims to prevent such behavior by reinforcing the correct goals, but alignment may not always succeed and can lead to unwanted side effects. We propose self-incrimination training, which inste…

• On Discovering Algorithms for Adversarial Imitation Learning

  Shashank Reddy Chirra, Jayden Teoh, Praveen Paruchuri, Pradeep Varakantham · 27 de febrero de 2026

  Adversarial Imitation Learning (AIL) methods, while effective in settings with limited expert demonstrations, are often considered unstable. These approaches typically decompose into two components: Density Ratio (DR) estimation $\frac{\rho_E}{\rho_{\pi}}$, where a discriminator estimates the relati…

• RaPA: Enhancing Transferable Targeted Attacks via Random Parameter Pruning

  Tongrui Su, Qingbin Li, Shengyu Zhu, Wei Chen, Xueqi Cheng · 27 de febrero de 2026

  Compared to untargeted attacks, targeted transfer-based attack is still suffering from much lower Attack Success Rates (ASRs), although significant improvements have been achieved by kinds of methods, such as diversifying input, stabilizing the gradient, and re-training surrogate models. In this pap…

• Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation

  Xiaosen Wang, Zhijin Ge, Bohan Liu, Zheng Fang, Fengfan Zhou, Ruixuan Zhang, Shaokang Wang, Yuyang Luo · 27 de febrero de 2026

  Adversarial transferability refers to the capacity of adversarial examples generated on the surrogate model to deceive alternate, unexposed victim models. This property eliminates the need for direct access to the victim model during an attack, thereby raising considerable security concerns in pract…

• Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models

  Guangnian Wan, Qi Li, Gongfan Fang, Xinyin Ma, Xinchao Wang · 27 de febrero de 2026

  Multimodal Diffusion Language Models (MDLMs) have recently emerged as a competitive alternative to their autoregressive counterparts. Yet their vulnerability to backdoor attacks remains largely unexplored. In this work, we show that well-established data-poisoning pipelines can successfully implant …

• ImpMIA: Leveraging Implicit Bias for Membership Inference Attack

  Yuval Golbari, Navve Wasserman, Gal Vardi, Michal Irani · 26 de febrero de 2026

  Determining which data samples were used to train a model, known as Membership Inference Attack (MIA), is a well-studied and important problem with implications on data privacy. SotA methods (which are black-box attacks) rely on training many auxiliary reference models to imitate the behavior of the…