Adversarial Robustness in Machine Learning

Neueste Paper

• Improving reproducibility by controlling random seed stability in machine learning based estimation via bagging

  Nicholas Williams, Alejandro Schuler · 21. April 2026

  Predictions from machine learning algorithms can vary across random seeds, inducing instability in downstream debiased machine learning estimators. We formalize random seed stability via a concentration condition and prove that subbagging guarantees stability for any bounded-outcome regression algor…

• Safe Control using Learned Safety Filters and Adaptive Conformal Inference

  Sacha Huriot, Ihab Tabbara, Hussein Sibai · 21. April 2026

  Safety filters have been shown to be effective tools to ensure the safety of control systems with unsafe nominal policies. To address scalability challenges in traditional synthesis methods, learning-based approaches have been proposed for designing safety filters for systems with high-dimensional s…

• SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models

  Noor Islam S. Mohammad, Ulu\u{g} Bayaz{\i}t · 21. April 2026

  Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains lacking. We present SafeLM, a framework that jointly addresses four pillars of LLM safety: privacy, security, misinformation, and adversarial robustne…

• Support Sufficiency as Consequence-Sensitive Compression in Belief Arbitration

  Mark Walsh · 21. April 2026

  When a system commits to a hypothesis, much of the evidential structure behind that commitment is lost to compression. Standard accounts assume that selected content and scalar confidence suffice for downstream control. This paper argues that they do not, and that determining what must survive compr…

• Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignment

  Yuxi Li, Yi Liu, Yuekang Li, Ling Shi, Gelei Deng, Shengquan Chen, Kailong Wang · 21. April 2026

  Large language models (LLMs) have revolutionized various applications, making robust safety alignment essential to prevent harmful outputs. Current safety alignment techniques, however, harbor inherent vulnerabilities due to their reliance on logit suppression. In this work, we identify critical log…

• Forget What Matters, Keep the Rest: Selective Unlearning of Informative Tokens

  Seunghee Koh, Sunghyun Baek, Youngdong Kim, Junmo Kim · 21. April 2026

  Unlearning in large language models (LLMs) has emerged as a promising safeguard against adversarial behaviors. When the forgetting loss is applied uniformly without considering token-level semantic importance, model utility can be unnecessarily degraded. Recent studies have explored token-wise loss …

• SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models

  Dongxin Guo, Jikun Wu, Siu Ming Yiu · 21. April 2026

  Safety alignment in large language models is remarkably shallow: it is concentrated in the first few output tokens and reversible by fine-tuning on as few as 100 adversarial examples. This fragility becomes critical in real-world deployment, where models undergo sequential adaptation across domains …

• Guardrails in Logit Space: Safety Token Regularization for LLM Alignment

  Thong Bach, Truyen Tran · 21. April 2026

  Fine-tuning well-aligned large language models (LLMs) on new domains often degrades their safety alignment, even when using benign datasets. Existing safety alignment techniques primarily focus on pretraining, leaving fine-tuned models vulnerable to behavioral shifts. In this work, we introduce safe…

• Towards E-Value Based Stopping Rules for Bayesian Deep Ensembles

  Emanuel Sommer, Rickmer Schulte, Sarah Deubner, Julius Kobialka, David R\"ugamer · 21. April 2026

  Bayesian Deep Ensembles (BDEs) represent a powerful approach for uncertainty quantification in deep learning, combining the robustness of Deep Ensembles (DEs) with flexible multi-chain MCMC. While DEs are affordable in most deep learning settings, (long) sampling of Bayesian neural networks can be p…

• Scalable Neighborhood-Based Multi-Agent Actor-Critic

  Tim Goppelsroeder, Rasmus Jensen · 21. April 2026

  We propose MADDPG-K, a scalable extension to Multi-Agent Deep Deterministic Policy Gradient (MADDPG) that addresses the computational limitations of centralized critic approaches. Centralized critics, which condition on the observations and actions of all agents, have demonstrated significant perfor…

• Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition

  Prasoon Goyal, Sattvik Sahai, Michael Johnston, Hangjie Shi, Yao Lu, Shaohua Liu, Anna Rumshisky, Rahul Gupta, Anna Gottardi, Desheng Zhang, Lavina Vaz, Leslie Ball, Lucy Hu, Luke Dai, Samyuth Sagi, Maureen Murray, Sankaranarayanan Ananthakrishnan · 21. April 2026

  Post-training Large Language Models requires diverse, high-quality data which is rare and costly to obtain, especially in low resource domains and for multi-turn conversations. Common solutions are crowdsourcing or synthetic generation, but both often yield low-quality or low-diversity data. We intr…

• Depth Registers Unlock W4A4 on SwiGLU: A Reader/Generator Decomposition

  Ziyang Liu · 21. April 2026

  We study post-training W4A4 quantization in a controlled 300M-parameter SwiGLU decoder-only language model trained on 5B tokens of FineWeb-Edu, and ask which input-activation sites dominate the error. Naive round-to-nearest W4A4 collapses validation perplexity from FP16 23.6 to 1727. A simple residu…

• LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users

  Almog Hilel, Riddhi Bhagwat, Idan Shenfeld, Jacob Andreas, Leshem Choshen · 21. April 2026

  We describe a vulnerability in language models (LMs) trained with user feedback, whereby a single user can persistently alter LM knowledge and behavior given only the ability to provide prompts and upvote / downvote feedback on LM outputs. To implement the attack, the attacker prompts the LM to stoc…

• The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability

  Prashant C. Raju · 21. April 2026

  Reliable deployment of language models requires two capabilities that appear distinct but share a common geometric foundation: predicting whether a model will accept targeted behavioral control, and detecting when its internal structure degrades. We show that geometric stability, the consistency of …

• Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)

  Tianbao Zhang · 21. April 2026

  Large Language Models (LLMs) produce a controllability gap in safety-critical engineering: even low rates of undetected constraint violations render a system undeployable. Current orchestration paradigms suffer from sycophantic compliance, context attention decay [Liu et al., 2024], and stochastic o…

• Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

  Jiachen Qian, Zhaolu Kang · 21. April 2026

  The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their adversarial robustness remains underexplored. We identify Visual Dominance Hallucination (VDH), where imperceptible visual cues can override textual …

• Towards Reliable Testing of Machine Unlearning

  Anna Mazhar, Sainyam Galhotra · 21. April 2026

  Machine learning components are now central to AI-infused software systems, from recommendations and code assistants to clinical decision support. As regulations and governance frameworks increasingly require deleting sensitive data from deployed models, machine unlearning is emerging as a practical…

• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning

  Fangyuan Liu, Sirui Zhao, Zeyu Zhang, Jinyang Huang, Feng-Qi Cui, Bin Luo, Tong Xu, Meng Li, Enhong Chen · 21. April 2026

  Automated depression estimation is highly vulnerable to signal corruption and ambient noise in real-world deployment. Prevailing deterministic methods produce uncalibrated point estimates, exposing safety-critical clinical systems to the severe risk of overconfident misdiagnoses. To establish a high…

• Continual Safety Alignment via Gradient-Based Sample Selection

  Thong Bach, Dung Nguyen, Thao Minh Le, Truyen Tran · 21. April 2026

  Large language models require continuous adaptation to new tasks while preserving safety alignment. However, fine-tuning on even benign data often compromises safety behaviors, including refusal of harmful requests, truthfulness, and commonsense reasoning. We investigate which training samples cause…

• Safety, Security, and Cognitive Risks in State-Space Models: A Systematic Threat Analysis with Spectral, Stateful, and Capacity Attacks

  Manoj Parmar · 21. April 2026

  State-Space Models (SSMs) -- structured SSMs (S4, S4D, DSS, S5), selective SSMs (Mamba, Mamba-2), and hybrid architectures (Jamba) -- are deployed in safety-critical long-context applications: genomic analysis, clinical time-series forecasting, and cybersecurity log processing. Their linear-time sca…

• SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamics

  Haolong Hu, Hanyu Li, Tiancheng He, Huahui Yi, An Zhang, Qiankun Li, Kun Wang, Yang Liu, Zhigang Zeng · 21. April 2026

  MLLMs are increasingly deployed in multi-turn settings, where attackers can escalate unsafe intent through the evolving visual-text history and exploit long-context safety decay. Yet safety alignment is still dominated by single-turn data and fixed-template dialogues, leaving a mismatch between trai…

• CCAR: Intrinsic Robustness as an Emergent Geometric Property

  Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri · 21. April 2026

  Standard supervised learning optimizes for predictive accuracy but remains agnostic to the internal geometry of learned features, often yielding representations that are entangled and brittle. We propose Class-Conditional Activation Regularization (CCAR) to explicitly engineer the feature space, imp…

• InfoChess: A Game of Adversarial Inference and a Laboratory for Quantifiable Information Control

  Kieran A. Murphy · 20. April 2026

  We propose InfoChess, a symmetric adversarial game that elevates competitive information acquisition to the primary objective. There is no piece capture, removing material incentives that would otherwise confound the role of information. Instead, pieces are used to alter visibility. Players are scor…

• Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation

  Yisheng Zhong, Sijia Liu, Zhuangdi Zhu · 20. April 2026

  Large Language Models (LLMs) unlearning is crucial for removing hazardous or privacy-leaking information from the model. Practical LLM unlearning demands satisfying multiple challenging objectives simultaneously: removing undesirable knowledge, preserving general utility, avoiding over-refusal of ne…

• Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs

  Wai Man Si, Mingjie Li, Michael Backes, Yang Zhang · 20. April 2026

  Machine learning models are increasingly deployed in real-world applications, but even aligned models such as Mistral and LLaVA still exhibit unsafe behaviors inherited from pre-training. Current alignment methods like SFT and RLHF primarily encourage models to generate preferred responses, but do n…