Adversarial Robustness in Machine Learning

Neueste Paper

• GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

  Xueyi Li, Zhuoneng Zhou, Zitao Liu, Yongdong Wu, Weiqi Luo · 3. Februar 2026

  Large language models (LLMs) have demonstrated remarkable potential for automatic short answer grading (ASAG), significantly boosting student assessment efficiency and scalability in educational scenarios. However, their vulnerability to adversarial manipulation raises critical concerns about automa…

• David vs. Goliath: Verifiable Agent-to-Agent Jailbreaking via Reinforcement Learning

  Samuel Nellessen, Tal Kachman · 3. Februar 2026

  The evolution of large language models into autonomous agents introduces adversarial failures that exploit legitimate tool privileges, transforming safety evaluation in tool-augmented environments from a subjective NLP task into an objective control problem. We formalize this threat model as Tag-Alo…

• Adversarial Reward Auditing for Active Detection and Mitigation of Reward Hacking

  Mohammad Beigi, Ming Jin, Junshan Zhang, Qifan Wang, Lifu Huang · 3. Februar 2026

  Reinforcement Learning from Human Feedback (RLHF) remains vulnerable to reward hacking, where models exploit spurious correlations in learned reward models to achieve high scores while violating human intent. Existing mitigations rely on static defenses that cannot adapt to novel exploitation strate…

• Bypassing Prompt Injection Detectors through Evasive Injections

  Md Jahedur Rahman, Ihsen Alouani · 3. Februar 2026

  Large language models (LLMs) are increasingly used in interactive and retrieval-augmented systems, but they remain vulnerable to task drift; deviations from a user's intended instruction due to injected secondary prompts. Recent work has shown that linear probes trained on activation deltas of LLMs'…

• Efficient Adversarial Attacks on High-dimensional Offline Bandits

  Seyed Mohammad Hadi Hosseini, Amir Najafi, Mahdieh Soleymani Baghshah · 3. Februar 2026

  Bandit algorithms have recently emerged as a powerful tool for evaluating machine learning models, including generative image models and large language models, by efficiently identifying top-performing candidates without exhaustive comparisons. These methods typically rely on a reward model, often d…

• Your AI-Generated Image Detector Can Secretly Achieve SOTA Accuracy, If Calibrated

  Muli Yang, Gabriel James Goenawan, Henan Wang, Huaiyuan Qin, Chenghao Xu, Yanhua Yang, Fen Fang, Ying Sun, Joo-Hwee Lim, Hongyuan Zhu · 3. Februar 2026

  Despite being trained on balanced datasets, existing AI-generated image detectors often exhibit systematic bias at test time, frequently misclassifying fake images as real. We hypothesize that this behavior stems from distributional shift in fake samples and implicit priors learned during training. …

• Assessing Domain-Level Susceptibility to Emergent Misalignment from Narrow Finetuning

  Abhishek Mishra, Mugilan Arulvanan, Reshma Ashok, Polina Petrova, Deepesh Suranjandass, Donnie Winkelmann · 3. Februar 2026

  Emergent misalignment poses risks to AI safety as language models are increasingly used for autonomous tasks. In this paper, we present a population of large language models (LLMs) fine-tuned on insecure datasets spanning 11 diverse domains, evaluating them both with and without backdoor triggers on…

• MACD: Model-Aware Contrastive Decoding via Counterfactual Data

  Qixin Xiao, Kun Zhou · 3. Februar 2026

  Video language models (Video-LLMs) are prone to hallucinations, often generating plausible but ungrounded content when visual evidence is weak, ambiguous, or biased. Existing decoding methods, such as contrastive decoding (CD), rely on random perturbations to construct contrastive data for mitigatin…

• ReasoningBomb: A Stealthy Denial-of-Service Attack by Inducing Pathologically Long Reasoning in Large Reasoning Models

  Xiaogeng Liu, Xinyan Wang, Yechao Zhang, Sanjay Kariyappa, Chong Xiang, Muhao Chen, G. Edward Suh, Chaowei Xiao · 3. Februar 2026

  Large reasoning models (LRMs) extend large language models with explicit multi-step reasoning traces, but this capability introduces a new class of prompt-induced inference-time denial-of-service (PI-DoS) attacks that exploit the high computational cost of reasoning. We first formalize inference cos…

• Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity

  Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi · 3. Februar 2026

  Large language models (LLMs) are known to "hallucinate" by generating false or misleading outputs. Hallucinations pose various harms, from erosion of trust to widespread misinformation. Existing hallucination evaluation, however, focuses only on correctness and often overlooks consistency, necessary…

• On the Fragility of AI-Based Channel Decoders under Small Channel Perturbations

  Haoyu Lei, Mohammad Jalali, Chin Wa Lau, Farzan Farnia · 3. Februar 2026

  Recent advances in deep learning have led to AI-based error correction decoders that report empirical performance improvements over traditional belief-propagation (BP) decoding on AWGN channels. While such gains are promising, a fundamental question remains: where do these improvements come from, an…

• AIRE-Prune: Asymptotic Impulse-Response Energy for State Pruning in State Space Models

  Apurba Prasad Padhy, Fernando Camacho, Saibal Mukhopadhyay · 3. Februar 2026

  State space models (SSMs) often sacrifice capacity, search space, or stability to offset the memory and compute costs of large state dimensions. We introduce a structured post-training pruning method for SSMs -- AIRE-Prune (Asymptotic Impulse-Response Energy for State PRUN(E)) -- that reduces each l…

• Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation

  Seonghyeon Park, Jewon Yeom, Jaewon Sok, Jeongjae Park, Heejun Kim, Taesup Kim · 3. Februar 2026

  Quantifying uncertainty in Large Language Models (LLMs) is essential for mitigating hallucinations and enabling risk-aware deployment in safety-critical tasks. However, estimating Epistemic Uncertainty(EU) via Deep Ensembles is computationally prohibitive at the scale of modern models. We propose a …

• Provable Defense Framework for LLM Jailbreaks via Noise-Augumented Alignment

  Zehua Cheng, Jianwei Yang, Wei Dai, Jiahao Sun · 3. Februar 2026

  Large Language Models (LLMs) remain vulnerable to adaptive jailbreaks that easily bypass empirical defenses like GCG. We propose a framework for certifiable robustness that shifts safety guarantees from single-pass inference to the statistical stability of an ensemble. We introduce Certified Semanti…

• LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion

  Guanghao Zhou, Panjia Qiu, Cen Chen, Hongyu Li, Mingyuan Chu, Xin Zhang, Jun Zhou · 3. Februar 2026

  The safety mechanisms of large language models (LLMs) exhibit notable fragility, as even fine-tuning on datasets without harmful content may still undermine their safety capabilities. Meanwhile, existing safety alignment methods predominantly rely on the fine-tuning process, which inadvertently lead…

• The Illusion of Forgetting: Attack Unlearned Diffusion via Initial Latent Variable Optimization

  Manyi Li, Yufan Liu, Lai Jiang, Bing Li, Yuming Li, Weiming Hu · 3. Februar 2026

  Although unlearning-based defenses claim to purge Not-Safe-For-Work (NSFW) concepts from diffusion models (DMs), we reveals that this "forgetting" is largely an illusion. Unlearning partially disrupts the mapping between linguistic symbols and the underlying knowledge, which remains intact as dorman…

• Self-Guard: Defending Large Reasoning Models via enhanced self-reflection

  Jingnan Zheng, Jingjun Xu, Yanzhen Luo, Chenhang Cui, Gelei Deng, Zhenkai Liang, Xiang Wang, An Zhang, Tat-Seng Chua · 3. Februar 2026

  The emergence of Large Reasoning Models (LRMs) introduces a new paradigm of explicit reasoning, enabling remarkable advances yet posing unique risks such as reasoning manipulation and information leakage. To mitigate these risks, current alignment strategies predominantly rely on heavy post-training…

• MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety

  Xiaoyu Wen, Zhida He, Han Qi, Ziyu Wan, Zhongtian Ma, Ying Wen, Tianhang Zheng, Xingcheng Xu, Chaochao Lu, Qiaosheng Zhang · 3. Februar 2026

  Ensuring robust safety alignment is crucial for Large Language Models (LLMs), yet existing defenses often lag behind evolving adversarial attacks due to their \textbf{reliance on static, pre-collected data distributions}. In this paper, we introduce \textbf{MAGIC}, a novel multi-turn multi-agent rei…

• Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuron

  Sicheng Shen, Mingyang Lv, Han Shen, Jialin Wu, Binghao Wang, Zhou Yang, Guobin Shen, Dongcheng Zhao, Feifei Zhao, Yi Zeng · 3. Februar 2026

  The safety of large language models (LLMs) has increasingly emerged as a fundamental aspect of their development. Existing safety alignment for LLMs is predominantly achieved through post-training methods, which are computationally expensive and often fail to generalize well across different models.…

• Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Models

  Yingsha Xie, Tiansheng Huang, Enneng Yang, Rui Min, Wenjie Lu, Xiaochun Cao, Naiqiang Tan, Li Shen · 3. Februar 2026

  Safety alignment incurs safety tax that perturbs a large reasoning model's (LRM) general reasoning ability. Existing datasets used for safety alignment for an LRM are usually constructed by distilling safety reasoning traces and answers from an external LRM or human labeler. However, such reasoning …

• Text is All You Need for Vision-Language Model Jailbreaking

  Yihang Chen, Zhao Xu, Youyuan Jiang, Tianle Zheng, Cho-Jui Hsieh · 3. Februar 2026

  Large Vision-Language Models (LVLMs) are increasingly equipped with robust safety safeguards to prevent responses to harmful or disallowed prompts. However, these defenses often focus on analyzing explicit textual inputs or relevant visual scenes. In this work, we introduce Text-DJ, a novel jailbrea…

• Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models

  Wenbin Xing, Quanxing Zha, Lizheng Zu, Mengran Li, Ming Li, Junchi Yan · 3. Februar 2026

  Current research on video hallucination mitigation primarily focuses on isolated error types, leaving compositional hallucinations, arising from incorrect reasoning over multiple interacting spatial and temporal factors largely underexplored. We introduce OmniVCHall, a benchmark designed to systemat…

• Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models

  Kaiyuan Cui, Yige Li, Yutao Wu, Xingjun Ma, Sarah Erfani, Christopher Leckie, Hanxun Huang · 3. Februar 2026

  Vision-language models (VLMs) extend large language models (LLMs) with vision encoders, enabling text generation conditioned on both images and text. However, this multimodal integration expands the attack surface by exposing the model to image-based jailbreaks crafted to induce harmful responses. E…

• Statistical MIA: Rethinking Membership Inference Attack for Reliable Unlearning Auditing

  Jialong Sun, Zeming Wei, Jiaxuan Zou, Jiacheng Gong, Guanheng Wang, Chengyang Dong, Jialong Li, Bo Liu · 3. Februar 2026

  Machine unlearning (MU) is essential for enforcing the right to be forgotten in machine learning systems. A key challenge of MU is how to reliably audit whether a model has truly forgotten specified training data. Membership Inference Attacks (MIAs) are widely used for unlearning auditing, where sam…

• DuFFin: A Dual-Level Fingerprinting Framework for LLMs IP Protection

  Yuliang Yan, Haochun Tang, Shuo Yan, Enyan Dai · 3. Februar 2026

  Large language models (LLMs) are considered valuable Intellectual Properties (IP) for legitimate owners due to the enormous computational cost of training. It is crucial to protect the IP of LLMs from malicious stealing or unauthorized deployment. Despite existing efforts in watermarking and fingerp…