Adversarial Robustness in Machine Learning

Últimos artículos

• Data Leakage in Automotive Perception: Practitioners' Insights

  Md Abu Ahammed Babu, Sushant Kumar Pandey, Darko Durisic, Andras Balint, Miroslaw Staron · 9 de abril de 2026

  Data leakage is the inadvertent transfer of information between training and evaluation datasets that poses a subtle, yet critical, risk to the reliability of machine learning (ML) models in safety-critical systems such as automotive perception. While leakage is widely recognized in research, little…

• TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories

  Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang, Yun-Nung Chen · 9 de abril de 2026

  As large language models (LLMs) evolve from static chatbots into autonomous agents, the primary vulnerability surface shifts from final outputs to intermediate execution traces. While safety guardrails are well-benchmarked for natural language responses, their efficacy remains largely unexplored wit…

• Variational Feature Compression for Model-Specific Representations

  Zinan Guo, Zihan Wang, Chuan Yan, Liuhuo Wan, Ethan Ma, Guangdong Bai · 9 de abril de 2026

  As deep learning inference is increasingly deployed in shared and cloud-based settings, a growing concern is input repurposing, in which data submitted for one task is reused by unauthorized models for another. Existing privacy defenses largely focus on restricting data access, but provide limited c…

• Adversarial Robustness of Time-Series Classification for Crystal Collimator Alignment

  Xaver Fink, Borja Fernandez Adiego, Daniele Mirarchi, Eloise Matheson, Alvaro Garcia Gonzales, Gianmarco Ricci, Joost-Pieter Katoen · 9 de abril de 2026

  In this paper, we analyze and improve the adversarial robustness of a convolutional neural network (CNN) that assists crystal-collimator alignment at CERN's Large Hadron Collider (LHC) by classifying a beam-loss monitor (BLM) time series during crystal rotation. We formalize a local robustness prope…

• Steering the Verifiability of Multimodal AI Hallucinations

  Jianhong Pang, Ruoxi Cheng, Ziyi Ye, Xingjun Ma, Zuxuan Wu, Xuanjing Huang, Yu-Gang Jiang · 9 de abril de 2026

  AI applications driven by multimodal large language models (MLLMs) are prone to hallucinations and pose considerable risks to human users. Crucially, such hallucinations are not equally problematic: some hallucination contents could be detected by human users(i.e., obvious hallucinations), while oth…

• Towards Accurate and Calibrated Classification: Regularizing Cross-Entropy From A Generative Perspective

  Qipeng Zhan, Zhuoping Zhou, Li Shen · 9 de abril de 2026

  Accurate classification requires not only high predictive accuracy but also well-calibrated confidence estimates. Yet, modern deep neural networks (DNNs) are often overconfident, primarily due to overfitting on the negative log-likelihood (NLL). While focal loss variants alleviate this issue, they t…

• VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts

  Peigui Qi, Kunsheng Tang, Yanpu Yu, Jialin Wu, Yide Song, Wenbo Zhou, Zhicong Huang, Cheng Hong, Weiming Zhang, Nenghai Yu · 9 de abril de 2026

  Vision-Language Models (VLMs) face significant safety vulnerabilities from malicious prompt attacks due to weakened alignment during visual integration. Existing defenses suffer from efficiency and robustness. To address these challenges, we first propose the Multimodal Aggregated Feature Extraction…

• Stealthy and Adjustable Text-Guided Backdoor Attacks on Multimodal Pretrained Models

  Yiyang Zhang, Chaojian Yu, Ziming Hong, Yuanjie Shao, Qinmu Peng, Tongliang Liu, Xinge You · 8 de abril de 2026

  Multimodal pretrained models are vulnerable to backdoor attacks, yet most existing methods rely on visual or multimodal triggers, which are impractical since visually embedded triggers rarely occur in real-world data. To overcome this limitation, we propose a novel Text-Guided Backdoor (TGB) attack …

• Towards Reliable Forgetting: A Survey on Machine Unlearning Verification

  Lulu Xue, Shengshan Hu, Wei Lu, Yan Shen, Dongxu Li, Peijin Guo, Ziqi Zhou, Minghui Li, Yanjun Zhang, Leo Yu Zhang · 8 de abril de 2026

  With growing demands for privacy protection, security, and legal compliance (e.g., GDPR), machine unlearning has emerged as a critical technique for ensuring the controllability and regulatory alignment of machine learning models. However, a fundamental challenge in this field lies in effectively ve…

• Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems

  Anshul Pathak, Nishant Jain · 8 de abril de 2026

  Enterprise multi-agent AI systems produce thousands of inter-agent interactions per hour, yet existing observability tools capture these dependencies without enforcing anything. OpenTelemetry and Langfuse collect telemetry but treat governance as a downstream analytics concern, not a real-time enfor…

• FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment

  Daniel Kuznetsov, Ofir Cohen, Karin Shistik, Rami Puzis, Asaf Shabtai · 8 de abril de 2026

  Safety-aligned LLMs go through refusal training to reject harmful requests, but whether these mechanisms remain effective under emotionally charged stimuli is unexplored. We introduce FreakOut-LLM, a framework investigating whether emotional context compromises safety alignment in adversarial settin…

• AttnDiff: Attention-based Differential Fingerprinting for Large Language Models

  Haobo Zhang, Zhenhua Xu, Junxian Li, Shangfeng Sheng, Dezhang Kong, Meng Han · 8 de abril de 2026

  Protecting the intellectual property of open-weight large language models (LLMs) requires verifying whether a suspect model is derived from a victim model despite common laundering operations such as fine-tuning (including PPO/DPO), pruning/compression, and model merging. We propose \textsc{AttnDiff…

• HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models

  Reihaneh Zohrabi, Hosein Hasani, Akshita Gupta, Mahdieh Soleymani Baghshah, Anna Rohrbach, Marcus Rohrbach · 8 de abril de 2026

  Large vision-language models can produce object hallucinations in image descriptions, highlighting the need for effective detection and mitigation strategies. Prior work commonly relies on the model's attention weights on visual tokens as a detection signal. We reveal that coarse-grained attention-b…

• Simulating the Evolution of Alignment and Values in Machine Intelligence

  Jonathan Elsworth Eicher · 8 de abril de 2026

  Model alignment is currently applied in a vacuum, evaluated primarily through standardised benchmark performance. The purpose of this study is to examine the effects of alignment on populations of models through time. We focus on the treatment of beliefs which contain both an alignment signal (how w…

• XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts

  Jiahao Xu, Rui Hu, Olivera Kotevska, Zikai Zhang · 8 de abril de 2026

  Multi-bit watermarking has emerged as a promising solution for embedding imperceptible binary messages into Large Language Model (LLM)-generated text, enabling reliable attribution and tracing of malicious usage of LLMs. Despite recent progress, existing methods still face key limitations: some beco…

• MEC: Machine-Learning-Assisted Generalized Entropy Calibration for Semi-Supervised Mean Estimation

  Se Yoon Lee, Jae Kwang Kim · 8 de abril de 2026

  Obtaining high-quality labels is costly, whereas unlabeled covariates are often abundant, motivating semi-supervised inference methods with reliable uncertainty quantification. Prediction-powered inference (PPI) leverages a machine-learning predictor trained on a small labeled sample to improve effi…

• Swiss-Bench 003: Evaluating LLM Reliability and Adversarial Security for Swiss Regulatory Contexts

  Fatih Uenal · 8 de abril de 2026

  The deployment of large language models (LLMs) in Swiss financial and regulatory contexts demands empirical evidence of both production reliability and adversarial security, dimensions not jointly operationalized in existing Swiss-focused evaluation frameworks. This paper introduces Swiss-Bench 003 …

• Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification

  Courtney Franzen, Farhad Pourkamali-Anaraki · 8 de abril de 2026

  Neural network classifiers trained with cross-entropy loss achieve strong predictive accuracy but lack the capability to provide inherent predictive uncertainty estimates, thus requiring external techniques to obtain these estimates. In addition, softmax scores for the true class can vary substantia…

• Which Leakage Types Matter?

  Simon Roth · 7 de abril de 2026

  Twenty-eight within-subject counterfactual experiments across 2,047 tabular datasets, plus a boundary experiment on 129 temporal datasets, measuring the severity of four data leakage classes in machine learning. Class I (estimation - fitting scalers on full data) is negligible: all nine conditions p…

• Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification

  Samar Ansari · 7 de abril de 2026

  The governance of frontier AI increasingly relies on controlling access to computational resources, yet the hardware-level mechanisms invoked by policy proposals remain largely unexamined from an engineering perspective. This paper bridges the gap between AI governance and computer engineering by pr…

• Explainability-Guided Adversarial Attacks on Transformer-Based Malware Detectors Using Control Flow Graphs

  Andrew Wheeler, Kshitiz Aryal, Maanak Gupta · 7 de abril de 2026

  Transformer-based malware detection systems operating on graph modalities such as control flow graphs (CFGs) achieve strong performance by modeling structural relationships in program behavior. However, their robustness to adversarial evasion attacks remains underexplored. This paper examines the vu…

• Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices

  Alexis Burgon, Berkman Sahiner, Nicholas A Petrick, Gene Pennello, Ravi K Samala · 7 de abril de 2026

  This work addresses challenges in evaluating adaptive artificial intelligence (AI) models for medical devices, where iterative updates to both models and evaluation datasets complicate performance assessment. We introduce a novel approach with three complementary measurements: learning (model improv…

• ClawSafety: "Safe" LLMs, Unsafe Agents

  Bowen Wei, Yunbei Zhang, Jinhao Pan, Kai Mei, Xiao Wang, Jihun Hamm, Ziwei Zhu, Yingqiang Ge · 7 de abril de 2026

  Personal AI agents like OpenClaw run with elevated privileges on users' local machines, where a single successful prompt injection can leak credentials, redirect financial transactions, or destroy files. This threat goes well beyond conventional text-level jailbreaks, yet existing safety evaluations…

• Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models

  Sohyeon Kim, Sang Yeon Yoon, Kyeongbo Kong · 7 de abril de 2026

  Large Vision-Language Models (LVLMs) have achieved impressive progress in multimodal reasoning, yet they remain prone to object hallucinations, generating descriptions of objects that are not present in the input image. Recent approaches attempt to mitigate hallucinations by suppressing unreliable v…

• TraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol

  Khanh Linh Nguyen, Hoa Nghiem, Tu Tran · 7 de abril de 2026

  AI control protocols use monitors to detect attacks by untrusted AI agents, but standard single-score monitors face two limitations: they miss subtle attacks where outputs look clean but reasoning is off, and they collapse to near-zero safety when the monitor is the same model as the agent (collusio…