Adversarial Robustness in Machine Learning

Neueste Paper

• AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attacks

  Weiming Song, Xuan Xie, Ruiping Yin · 17. Februar 2026

  Large language models (LLMs) remain vulnerable to jailbreak prompts that elicit harmful or policy-violating outputs, while many existing defenses rely on expensive fine-tuning, intrusive prompt rewriting, or external guardrails that add latency and can degrade helpfulness. We present AISA, a lightwe…

• Nonparametric Distribution Regression Re-calibration

  \'Ad\'am Jung, Domokos M. Kelen, Andr\'as A. Bencz\'ur · 17. Februar 2026

  A key challenge in probabilistic regression is ensuring that predictive distributions accurately reflect true empirical uncertainty. Minimizing overall prediction error often encourages models to prioritize informativeness over calibration, producing narrow but overconfident predictions. However, in…

• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents

  Xu Li, Simon Yu, Minzhou Pan, Yiyou Sun, Bo Li, Dawn Song, Xue Lin, Weiyan Shi · 17. Februar 2026

  LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This gap widens as agents engage in multi-turn interactions and employ diverse tools, introducing new risks overlooked by existing benchmarks. To systematica…

• Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems

  Robab Aghazadeh Chakherlou, Siddartha Khastgir, Xingyu Zhao, Jerein Jeyachandran, Shufeng Chen · 17. Februar 2026

  Assuring the trustworthiness and safety of AI systems, e.g., autonomous vehicles (AV), depends critically on the data-related safety properties, e.g., representativeness, completeness, etc., of the datasets used for their training and testing. Among these properties, this paper focuses on representa…

• Computable Bernstein Certificates for Cross-Fitted Clipped Covariance Estimation

  Even He, Zaizai Yan · 17. Februar 2026

  We study operator-norm covariance estimation from heavy-tailed samples that may include a small fraction of arbitrary outliers. A simple and widely used safeguard is \emph{Euclidean norm clipping}, but its accuracy depends critically on an unknown clipping level. We propose a cross-fitted clipped co…

• Privacy-Concealing Cooperative Perception for BEV Scene Segmentation

  Song Wang, Lingling Li, Marcus Santos, Guanghui Wang · 17. Februar 2026

  Cooperative perception systems for autonomous driving aim to overcome the limited perception range of a single vehicle by communicating with adjacent agents to share sensing information. While this improves perception performance, these systems also face a significant privacy-leakage issue, as sensi…

• NEST: Nascent Encoded Steganographic Thoughts

  Artem Karpov · 17. Februar 2026

  Monitoring chain-of-thought (CoT) reasoning is a foundational safety technique for large language model (LLM) agents; however, this oversight is compromised if models learn to conceal their reasoning. We explore the potential for steganographic CoT -- where models hide secret reasoning within innocu…

• CAIRO: Decoupling Order from Scale in Regression

  Harri Vanhems, Yue Zhao, Peng Shi, Archer Y. Yang · 17. Februar 2026

  Standard regression methods typically optimize a single pointwise objective, such as mean squared error, which conflates the learning of ordering with the learning of scale. This coupling renders models vulnerable to outliers and heavy-tailed noise. We propose CAIRO (Calibrate After Initial Rank Ord…

• Backdooring Bias in Large Language Models

  Anudeep Das, Prach Chantasantitam, Gurjot Singh, Lipeng He, Mariia Ponomarenko, Florian Kerschbaum · 17. Februar 2026

  Large language models (LLMs) are increasingly deployed in settings where inducing a bias toward a certain topic can have significant consequences, and backdoor attacks can be used to produce such models. Prior work on backdoor attacks has largely focused on a black-box threat model, with an adversar…

• Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks

  Lukas Struppek, Adam Gleave, Kellin Pelrine · 17. Februar 2026

  As the capabilities of large language models continue to advance, so does their potential for misuse. While closed-source models typically rely on external defenses, open-weight models must primarily depend on internal safeguards to mitigate harmful behavior. Prior red-teaming research has largely f…

• Optimized Certainty Equivalent Risk-Controlling Prediction Sets

  Jiayi Huang, Amirmohammad Farzaneh, Osvaldo Simeone · 17. Februar 2026

  In safety-critical applications such as medical image segmentation, prediction systems must provide reliability guarantees that extend beyond conventional expected loss control. While risk-controlling prediction sets (RCPS) offer probabilistic guarantees on the expected risk, they fail to capture ta…

• In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes

  Trishit Mondal, Ameya D. Jagtap · 17. Februar 2026

  Transformer architectures have revolutionized machine learning across a wide range of domains, from natural language processing to scientific computing. However, their growing deployment in high-stakes applications, such as computer vision, natural language processing, healthcare, autonomous systems…

• Boundary Point Jailbreaking of Black-Box LLMs

  Xander Davies, Giorgi Giglemiani, Edmund Lau, Eric Winsor, Geoffrey Irving, Yarin Gal · 17. Februar 2026

  Frontier LLMs are safeguarded against attempts to extract harmful information via adversarial prompts known as "jailbreaks". Recently, defenders have developed classifier-based systems that have survived thousands of hours of human red teaming. We introduce Boundary Point Jailbreaking (BPJ), a new c…

• Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

  Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng · 17. Februar 2026

  Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-language rubrics and validated on benchmarks. We identify a previously under-recognized vulnerability in this workflow, which we term Rubric-Induced Preference Drif…

• Image Can Bring Your Memory Back: A Novel Multi-Modal Guided Attack against Image Generation Model Unlearning

  Renyang Liu, Guanlin Li, Tianwei Zhang, See-Kiong Ng · 17. Februar 2026

  Recent advances in image generation models (IGMs), particularly diffusion-based architectures such as Stable Diffusion (SD), have markedly enhanced the quality and diversity of AI-generated visual content. However, their generative capability has also raised significant ethical, legal, and societal …

• The Rogue Scalpel: Activation Steering Compromises LLM Safety

  Anton Korznikov, Andrey Galichin, Alexey Dontsov, Oleg Y. Rogov, Ivan Oseledets, Elena Tutubalina · 17. Februar 2026

  Activation steering is a promising technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference. It is often framed as a precise, interpretable, and potentially safer alternative to fine-tuning. We demonstrate the opposite: s…

• Agentic AI for Commercial Insurance Underwriting with Adversarial Self-Critique

  Joyjit Roy, Samaresh Kumar Singh · 17. Februar 2026

  Commercial insurance underwriting is a labor-intensive process that requires manual review of extensive documentation to assess risk and determine policy pricing. While AI offers substantial efficiency improvements, existing solutions lack comprehensive reasoning capabilities and internal mechanisms…

• Formal Reasoning About Confidence and Automated Verification of Neural Networks

  Mohammad Afzal, S. Akshay, Blaise Genest, Ashutosh Gupta · 17. Februar 2026

  In the last decade, a large body of work has emerged on robustness of neural networks, i.e., checking if the decision remains unchanged when the input is slightly perturbed. However, most of these approaches ignore the confidence of a neural network on its output. In this work, we aim to develop a g…

• Synthesizing the Kill Chain: A Zero-Shot Framework for Target Verification and Tactical Reasoning on the Edge

  Jesse Barkley, Abraham George, Amir Barati Farimani · 17. Februar 2026

  Deploying autonomous edge robotics in dynamic military environments is constrained by both scarce domain-specific training data and the computational limits of edge hardware. This paper introduces a hierarchical, zero-shot framework that cascades lightweight object detection with compact Vision-Lang…

• Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges

  Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin, Prasant Mohapatra · 16. Februar 2026

  Agentic AI systems powered by large language models (LLMs) and endowed with planning, tool use, memory, and autonomy, are emerging as powerful, flexible platforms for automation. Their ability to autonomously execute tasks across web, software, and physical environments creates new and amplified sec…

• Extending confidence calibration to generalised measures of variation

  Andrew Thompson, Vivek Desai · 16. Februar 2026

  We propose the Variation Calibration Error (VCE) metric for assessing the calibration of machine learning classifiers. The metric can be viewed as an extension of the well-known Expected Calibration Error (ECE) which assesses the calibration of the maximum probability or confidence. Other ways of me…

• Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness

  Elie Attias, Cengiz Pehlevan, Dina Obeid · 16. Februar 2026

  Convolutional Neural Networks (CNNs) excel in many visual tasks but remain susceptible to adversarial attacks-imperceptible perturbations that degrade performance. Prior research reveals that brain-inspired regularizers, derived from neural recordings, can bolster CNN robustness; however, reliance o…

• Chimera: Neuro-Symbolic Attention Primitives for Trustworthy Dataplane Intelligence

  Rong Fu, Wenxin Zhang, Xiaowen Ma, Kun Liu, Wangyu Wu, Ziyu Kong, Jia Yee Tan, Tailong Luo, Xianda Li, Zeli Su, Youjin Wang, Yongtai Liu, Simon Fong · 16. Februar 2026

  Deploying expressive learning models directly on programmable dataplanes promises line-rate, low-latency traffic analysis but remains hindered by strict hardware constraints and the need for predictable, auditable behavior. Chimera introduces a principled framework that maps attention-oriented neura…

• TensorCommitments: A Lightweight Verifiable Inference for Language Models

  Oguzhan Baser, Elahe Sadeghi, Eric Wang, David Ribeiro Alves, Sam Kazemian, Hong Kang, Sandeep P. Chinchali, Sriram Vishwanath · 16. Februar 2026

  Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, and must trust that the remote GPU executes the LLM without any adversarial tampering. We critically ask how to achieve verifiable LLM inference, where a prover (the service) must convince a verifier (t…

• TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios

  Wentao Xu, Zhongming Yao, Weihao Li, Zhenghang Song, Yumeng Song, Tianyi Li, Yushuai Li · 16. Februar 2026

  Constrained Reinforcement Learning (CRL) aims to optimize decision-making policies under constraint conditions, making it highly applicable to safety-critical domains such as autonomous driving, robotics, and power grid management. However, existing robust CRL approaches predominantly focus on singl…