Adversarial Robustness in Machine Learning

Neueste Paper

• PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits

  Neeladri Bhuiya, Madhav Aggarwal, Diptanshu Purwar · 9. Juni 2026

  Large Language Models (LLMs) are improving at an exceptional rate. With the advent of agentic workflows, multi-turn dialogue has become the de facto mode of interaction with LLMs for completing long and complex tasks. While LLM capabilities continue to improve, they remain increasingly susceptible t…

• Pretrained, Frozen, Still Leaking: Auditing Cross-Encoder Attribute Transfer in EEG Foundation Models

  Jianwei Tai · 9. Juni 2026

  EEG foundation-model releases are usually audited one endpoint at a time: raw-reconstruction, membership inference, identity linkage, or DP-SGD on the downstream head. We audit the same released embeddings under all four endpoints jointly, on BIOT, LaBraM, and EEGPT, and show that each single-endpoi…

• Scaling Neural Network Verification with Tensor Parallelism and Fully Sharded Data Parallelism

  Sergei Vorobyov, Eugene Ilyushin · 9. Juni 2026

  Formal neural network verification -- proving that a network satisfies safety properties for \emph{all} inputs in a specified domain -- is bounded in practice by GPU memory: standard implementations of bound-propagation algorithms (IBP, CROWN, $\alpha$-CROWN) require weight and relaxation-coefficien…

• Hybrid Robustness Verification for Spatio-Temporal Neural Networks

  Sherwin Varghese, Matthew Wicker, Alessio Lomuscio · 9. Juni 2026

  With AI increasingly deployed in safety-critical systems, providing formal robustness guarantees for the underlying models is essential. Existing verification methods either rely on overly conservative approximations or incur prohibitive computational costs. For example, the use of lp-norm perturbat…

• Culturally-Adapted Red-Teaming Across East and Southeast Asian Contexts: A Methodological and Comparative Analysis

  Hyeji Choi, Yongtaek Lim, Minwoo Kim · 9. Juni 2026

  Multilingual safety evaluation of large language models (LLMs) has predominantly relied on direct translation (DT) of English benchmarks into target languages - an approach that converts surface-level linguistic form while failing to reflect the cultural context embedded in threat scenarios, social …

• SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agents

  Tanush Swaminathan, Runmin Jiang, Letian Zhang, Min Xu · 9. Juni 2026

  LLM-based scientific agents have shown strong capacity for autonomous research, yet their safety layers remain structurally divorced from core reasoning: they inspect pipeline outputs rather than shaping the deliberation that produces them. This separation opens two failure modes: safety signals acc…

• DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment

  Yi Nian, Tiankai Yang, Yudi Zhang, Qi Pan, Zelong Xu, Shenzhe Zhu, Qingqing Luan, Yue Huang, Xiangliang Zhang, Yue Zhao · 9. Juni 2026

  Safety alignment for large language models relies on preference data, but current pipelines often train on large, redundant datasets. Existing data selection methods typically score each preference pair independently, collapsing directional preference information into scalar quality or diversity sco…

• Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks

  Haoming Wen, Shi Chen, Qingyu Shi, Siyuan Liu, Minrui Luo, Jingzhao Zhang, Tianxing He · 9. Juni 2026

  Current open-weight large language models (LLMs) are prone to malicious finetuning attacks, which could compromise the safety alignment of LLMs with only a few steps of supervised finetuning (SFT) on poisoned datasets. Existing alignment-stage defenses are primarily designed to defend against attack…

• Targeting World Models to Compromise Robot Learning Pipelines

  Ethan Rathbun, Ahmed Agha, Saaduddin Mahmud, Christopher Amato, Alina Oprea, Eugene Bagdasarian · 9. Juni 2026

  World models have recently seen a rapid growth in both their popularity and capability as more data efficient tools for generating robot training data or simulating real world environments, with many works proposing their integration into the robot learning pipeline. While highly practical, in this …

• Safe-RULE: Safe Reinforcement UnLEarning

  Shixiong Jiang, Taozheng Zhu, Fanxin Kong · 9. Juni 2026

  Offline safe reinforcement learning (Safe RL) enables policy learning without online interactions, making it suitable for safety-critical systems such as robotics systems. However, its reliance on static datasets exposes offline Safe RL to data poisoning attacks, where adversaries inject malicious s…

• When Behavioral Safety Evaluation Fails: A Representation-Level Perspective

  Enyi Jiang, Anders Gj{\o}lbye, Yibo Jacky Zhang, Sanmi Koyejo · 9. Juni 2026

  Large Language Model (LLM) safety has often been evaluated at the behavior level, which provides limited evidence of internal robustness, as these evaluations target outputs rather than representation-level vulnerability under intervention. We formalize this discrepancy as the audit gap: the differe…

• Constrained Paraphrase Consistency for LLM Hallucination Detection

  Shanshan Lin, Dongsheng Hong, Sibo Ju, Chao Chen, Xi Zhang, Xiangwen Liao · 9. Juni 2026

  Large language models (LLMs) can generate factually inconsistent claims, motivating accurate and scalable hallucination detectors. Prior work largely enlarges training sets via synthesis or new annotations, introducing increasing cost and potential bias while underusing the consistency implied by se…

• Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models

  Sanjay Kariyappa, G. Edward Suh · 9. Juni 2026

  Reasoning language models deployed in agentic workflows must follow an instruction hierarchy: when instructions from different sources conflict, the model should obey the highest-privilege applicable instruction. Existing benchmarks largely measure this behavior end-to-end, asking whether the final …

• Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs

  Omar Mahmoud, Aly M. Kassem, Thommen George Karimpanal, Buddhika Laknath Semage, Negar Rostamzadeh, Golnoosh Farnadi, Santu Rana · 9. Juni 2026

  Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to specific triggers or behaviors. We show this view is incomplete. Across diverse backdoor behaviors, we identify a shared latent mechanism that can be detected, ca…

• BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models

  Naveen Bera, Pulijala Sai Nikhila, Kondaguduru Abhiram, Shaik Gayaz Ali, Shoaib Sadiq Salehmohamed, Shaik Mohammed Omar, Jinal Prashant Thakkar, Hansika Aredla, Shalmali Ayachit · 9. Juni 2026

  Hallucination in large language models (LLMs), defined as the generation of factually incorrect or unsupported content, remains a critical barrier to reliable deployment. We present BEACON (Behavioral Entropy Aggregation for Cross-model hallucination detectiON), a black-box hallucination detection f…

• Human-Centred Risk Mitigation for AI-Mediated Information Manipulation: A SOCMINT Framework Based on Information Manipulation Sets

  Antonio Scala · 9. Juni 2026

  AI-mediated information manipulation increasingly takes the form of social cyber attacks that target trust, attention, credibility, reputation, and decision-making rather than only technical infrastructures or isolated false contents. Existing defensive approaches often oscillate between incident-le…

• A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers

  Hannah Gao (Massachusetts Institute of Technology), Isha Agarwal (Massachusetts Institute of Technology), Dylan Hadfield-Menell (Massachusetts Institute of Technology), Rachel Ma (Massachusetts Institute of Technology) · 9. Juni 2026

  The widespread use of image classification models in high-risk, real-world situations necessitates making these models robust to slight disturbances or perturbations, such as blurring or sharpening, in the input images. While vision transformers (ViTs) play an integral role in many modern-day multi-…

• Training-Inference Kernel Contracts: Bounding Divergence in Post-Training and Deployment

  Bruce Changlong Xu, Lan Wu · 9. Juni 2026

  A modern post-training pipeline often writes one symbol for its policy, pi_theta, while evaluating it through two different programs: a training kernel optimized for autograd and an inference kernel optimized for low-precision, fused, dynamically batched serving. In finite precision, these kernels c…

• Model Multiplicity for Adversarial Detection in Small Language Model Training on Edge Devices

  Stefan Behfar, Richard Mortier · 9. Juni 2026

  The rise of edge-based machine learning has enabled distributed adaptation of language models across mobile and IoT devices, offering privacy preservation and real-time responsiveness. However, distributed fine-tuning of language models on untrusted or heterogeneous edge nodes introduces new vulnera…

• Harness Engineering for Physical AI: Robot Middleware Is the Harness Layer

  Sanghoon Lee, Jiyeong Chae, Kyung-Joon Park · 9. Juni 2026

  Robot middleware faces a new role in the era of Physical AI. Learned policies, planners, and vision-language-action (VLA) models now enter deployed robots as causal participants on the control path, but the layer that integrates them with timing, scheduling, and network has not been named. Recent la…

• SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?

  Rishi Desai, Jesse Hu, Joan Cabezas, Neel Harsola, Pratyush Shukla, Roey Ben Chaim, Adnan El Assadi, Omkaar Mukund Kamath, Fenil Faldu, Prannay Hebbar, Jiankai Sun, Yiyuan Li, Pramod Srinivasan, Ishan Gupta, Christopher Settles, Daniel Wang, Derek Chen, Pranav Raja, Albert Liu, Marek \v{S}uppa, Nevasini Sasikumar, Luyang Kong, Erik Quintanilla, Xiangyi Li, Ivan Bercovich, Steven Dillmann · 9. Juni 2026

  AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments. Yet current agent benchmarks largely evaluate short-form tasks, such as single pull requests, small tickets, or 5-10 minute exercises, limi…

• SecureClaw: Clawing Back Control of LLM Agents

  Yuhan Ma, Stefan Schmid · 9. Juni 2026

  Tool-using large language model (LLM) agents face two distinct security failures: unauthorized external actions and exposure of sensitive plaintext inside the runtime before any final output check can intervene. Existing defenses usually protect one boundary, either the planner/runtime or the action…

• Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges

  Yongtaek Lim, Hyeji Choi, Minwoo Kim · 9. Juni 2026

  Safety judges are increasingly deployed to evaluate model outputs against evolving criteria, yet recent meta-evaluation work shows they remain brittle under prompt and rubric variation, with false negative-rate swings of up to 0.24 reported for stylistic perturbations alone. We argue that safety jud…

• Semantic Quorum Assurance: Collective Certification for Non-Deterministic AI Infrastructure

  Jun He, Deying Yu · 9. Juni 2026

  As large language model (LLM) agents are integrated into autonomous cloud operations, distributed systems face a semantic reliability problem: proposer agents can generate production mutations, such as modifying IAM policies, opening firewall security groups, or executing data exports, that are synt…

• Developing Distance-Aware Physics-Constrained Probabilistic Frameworks for Industrial Prognostics

  Waleed Razzaq, Yun-Bo Zhao · 9. Juni 2026

  Development of reliable and physically interpretable probabilistic frameworks for industrial prognostics remain nascent, and existing literature is often insensitive as inputs move away from the training manifold. In this paper, we develop two sampling-free, distance-aware physics-constrained probab…