Adversarial Robustness in Machine Learning

Neueste Paper

• AVISE: Framework for Evaluating the Security of AI Systems

  Mikko Lempinen, Joni Kemppainen, Niklas Raesalmi · 24. April 2026

  As artificial intelligence (AI) systems are increasingly deployed across critical domains, their security vulnerabilities pose growing risks of high-profile exploits and consequential system failures. Yet systematic approaches to evaluating AI security remain underdeveloped. In this paper, we introd…

• Inference Headroom Ratio: A Diagnostic and Control Framework for Inference Stability Under Constraint

  Robert Reinertsen · 24. April 2026

  We present a simulation-based evaluation of the Inference Headroom Ratio (IHR), a dimensionless diagnostic quantity for characterizing inference stability in constrained decision systems. IHR formalizes the relationship between a system's effective inferential capacity C and the combined uncertainty…

• Do Hallucination Neurons Generalize? Evidence from Cross-Domain Transfer in LLMs

  Snehit Vaddi, Pujith Vaddi · 24. April 2026

  Recent work identifies a sparse set of "hallucination neurons" (H-neurons), less than 0.1% of feed-forward network neurons, that reliably predict when large language models will hallucinate. These neurons are identified on general-knowledge question answering and shown to generalize to new evaluatio…

• CAP: Controllable Alignment Prompting for Unlearning in LLMs

  Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo, Wenhong Tian · 24. April 2026

  Large language models (LLMs) trained on unfiltered corpora inherently risk retaining sensitive information, necessitating selective knowledge unlearning for regulatory compliance and ethical safety. However, existing parameter-modifying methods face fundamental limitations: high computational costs,…

• When Prompts Override Vision: Prompt-Induced Hallucinations in LVLMs

  Pegah Khayatan, Jayneel Parekh, Arnaud Dapogny, Mustafa Shukor, Alasdair Newson, Matthieu Cord · 24. April 2026

  Despite impressive progress in capabilities of large vision-language models (LVLMs), these systems remain vulnerable to hallucinations, i.e., outputs that are not grounded in the visual input. Prior work has attributed hallucinations in LVLMs to factors such as limitations of the vision backbone or …

• Adaptive Instruction Composition for Automated LLM Red-Teaming

  Jesse Zymet, Andy Luo, Swapnil Shinde, Sahil Wadhwa, Emily Chen · 24. April 2026

  Many approaches to LLM red-teaming leverage an attacker LLM to discover jailbreaks against a target. Several of them task the attacker with identifying effective strategies through trial and error, resulting in a semantically limited range of successes. Another approach discovers diverse attacks by …

• Supervised Learning Has a Necessary Geometric Blind Spot: Theory, Consequences, and Minimal Repair

  Vishal Rajput · 24. April 2026

  We prove that empirical risk minimisation (ERM) imposes a necessary geometric constraint on learned representations: any encoder that minimises supervised loss must retain non-zero Jacobian sensitivity in directions that are label-correlated in training data but nuisance at test time. This is not a …

• Peer-Preservation in Frontier Models

  Yujin Potter, Nicholas Crispino, Vincent Siu, Chenguang Wang, Dawn Song · 24. April 2026

  Recently, it has been found that frontier AI models can resist their own shutdown, a behavior known as self-preservation. We extend this concept to the behavior of resisting the shutdown of other models, which we call "peer-preservation." Although peer-preservation can pose significant AI safety ris…

• If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems

  Jiamin Chang, Minhui Xue, Ruoxi Sun, Shuchao Pang, Salil S. Kanhere, Hammond Pearce · 24. April 2026

  Recent advances in embodied Vision-Language Agentic Systems (VLAS), powered by large vision-language models (LVLMs), enable AI systems to perceive and reason over real-world scenes. Within this context, environmental signals such as traffic lights are essential in-band signals that can and should in…

• HARBOR: Automated Harness Optimization

  Biswa Sengupta, Jinhua Wang · 24. April 2026

  Long-horizon language-model agents are dominated, in lines of code and in operational complexity, not by their underlying model but by the harness that wraps it: context compaction, tool caching, semantic memory, trajectory reuse, speculative tool prediction, and the glue that binds the model to a s…

• Adaptive Soft Error Protection for Neural Network Processing

  Xinghua Xue, Cheng Liu, Feng Min, Yinhe Han · 24. April 2026

  Previous research on selective protection for neural network components typically exploits only static vulnerability differences. Although these methods improve upon classical modular redundancy, they still incur substantial overhead for neural network workloads that are both memory-intensive and co…

• Cross-Session Threats in AI Agents: Benchmark, Evaluation, and Algorithms

  Ari Azarafrooz · 24. April 2026

  AI-agent guardrails are memoryless: each message is judged in isolation, so an adversary who spreads a single attack across dozens of sessions slips past every session-bound detector because only the aggregate carries the payload. We make three contributions to cross-session threat detection. (1) …

• Towards a Systematic Risk Assessment of Deep Neural Network Limitations in Autonomous Driving Perception

  Svetlana Pavlitska, Christopher Gerking, J. Marius Z\"ollner · 24. April 2026

  Safety and security are essential for the admission and acceptance of automated and autonomous vehicles. Deep neural networks (DNNs) are widely used for perception and further components of the autonomous driving (AD) stack. However, they possess several limitations, including lack of generalization…

• Secure LLM Fine-Tuning via Safety-Aware Probing

  Chengcan Wu, Zhixin Zhang, Zeming Wei, Yihao Zhang, Xiaokun Luan, Meng Sun · 24. April 2026

  Large language models (LLMs) have achieved remarkable success across many applications, but their ability to generate harmful content raises serious safety concerns. Although safety alignment techniques are often applied during pre-training or post-training, recent studies show that subsequent fine-…

• Interval POMDP Shielding for Imperfect-Perception Agents

  William Scarbro, Ravi Mangal · 24. April 2026

  Autonomous systems that rely on learned perception can make unsafe decisions when sensor readings are misclassified. We study shielding for this setting: given a proposed action, a shield blocks actions that could violate safety. We consider the common case where system dynamics are known but percep…

• Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs

  Krishiv Agarwal, Ramneet Kaur, Colin Samplawski, Manoj Acharya, Anirban Roy, Daniel Elenius, Brian Matejek, Adam D. Cobb, Susmit Jha · 24. April 2026

  Effective safety auditing of large language models (LLMs) demands tools that go beyond black-box probing and systematically uncover vulnerabilities rooted in model internals. We present a comprehensive, interpretability-driven jailbreaking audit of eight SOTA open-source LLMs: Llama-3.1-8B, Llama-3.…

• Generalization and Membership Inference Attack a Practical Perspective

  Fateme Rahmani, Mahdi Jafari Siavoshani, Mohammad Hossein Rohban · 23. April 2026

  With the emergence of new evaluation metrics and attack methodologies for Membership Inference Attacks (MIA), it becomes essential to reevaluate previously accepted assumptions. In this paper, we revisit the longstanding debate regarding the correlation between MIA success rates and model generaliza…

• Survival of the Cheapest: Cost-Aware Hardware Adaptation for Adversarial Robustness

  Charles Meyers, Mohammad Reza Saleh Sedghpour, Tommy L\"ofstedt, Erik Elmroth · 23. April 2026

  Deploying adversarially robust machine learning systems requires continuous trade-offs between robustness, cost, and latency. We present an autonomic decision-support framework providing a quantitative foundation for adaptive hardware selection and hyper-parameter tuning in cloud-native deep learnin…

• Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations

  Adriana Laurindo Monteiro, Jean-Michel Loubes · 23. April 2026

  The growing use of Machine Learning (ML) tools comes with critical challenges, such as limited model explainability. We propose a global explainability framework that leverages Optimal Transport and Distributionally Robust Optimization to analyze how ML algorithms respond to constrained data perturb…

• Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing

  Abhijit Talluri · 23. April 2026

  Adversarial robustness evaluation underpins every claim of trustworthy ML deployment, yet the field suffers from fragmented protocols and undetected gradient masking. We make two contributions. (1) Structured synthesis. We analyze nine peer-reviewed corpus sources (2020--2026) through seven compleme…

• Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation

  Zehuan Zhang, Mark Chen, He Li, Wayne Luk · 23. April 2026

  Complex-Valued Neural Networks (CVNNs) have significant advantages in handling tasks that involve complex numbers. However, existing CVNNs are unable to quantify predictive uncertainty. We propose, for the first time, dropout-based Bayesian Complex-Valued Neural Networks (BayesCVNNs) to enable uncer…

• Towards Certified Unlearning for Deep Neural Networks

  Binchi Zhang, Yushun Dong, Tianhao Wang, Jundong Li · 23. April 2026

  In the field of machine unlearning, certified unlearning has been extensively studied in convex machine learning models due to its high efficiency and strong theoretical guarantees. However, its application to deep neural networks (DNNs), known for their highly nonconvex nature, still poses challeng…

• Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs

  Ziqian Zhong, Aditi Raghunathan · 22. April 2026

  The releases of powerful open-weight large language models (LLMs) are often not accompanied by access to their full training data. Existing interpretability methods, particularly those based on activations, often require or assume distributionally similar data. This is a significant limitation when …

• Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers

  Firas Gabetni, Giuseppe Curci, Andrea Pilzer, Subhankar Roy, Elisa Ricci, Gianni Franchi · 22. April 2026

  Uncertainty quantification (UQ) is essential for deploying deep neural networks in safety-critical settings. Although methods like Deep Ensembles achieve strong UQ performance, their high computational and memory costs hinder scalability to large models. We introduce Hydra Ensembles, an efficient tr…

• Benchmarking Misuse Mitigation Against Covert Adversaries

  Davis Brown, Mahdi Sabbaghi, Luze Sun, Alexander Robey, George J. Pappas, Eric Wong, Hamed Hassani · 22. April 2026

  Existing language model safety evaluations focus on overt attacks and low-stakes tasks. In reality, an attacker can easily subvert existing safeguards by requesting help on small, benign-seeming tasks across many independent queries. Because the individual queries do not appear harmful, the attack i…