Adversarial Robustness in Machine Learning

Últimos artículos

• Multilingual Hidden Prompt Injection Attacks on LLM-Based Academic Reviewing

  Panagiotis Theocharopoulos, Ajinkya Kulkarni, Mathew Magimai. -Doss · 30 de diciembre de 2025

  Large language models (LLMs) are increasingly considered for use in high-impact workflows, including academic peer review. However, LLMs are vulnerable to document-level hidden prompt injection attacks. In this work, we construct a dataset of approximately 500 real academic papers accepted to ICML a…

• DarkPatterns-LLM: A Multi-Layer Benchmark for Detecting Manipulative and Harmful AI Behavior

  Sadia Asif, Israel Antonio Rosales Laguan, Haris Khan, Shumaila Asif, Muneeb Asif · 30 de diciembre de 2025

  The proliferation of Large Language Models (LLMs) has intensified concerns about manipulative or deceptive behaviors that can undermine user autonomy, trust, and well-being. Existing safety benchmarks predominantly rely on coarse binary labels and fail to capture the nuanced psychological and social…

• DECEPTICON: How Dark Patterns Manipulate Web Agents

  Phil Cuvin, Hao Zhu, Diyi Yang · 30 de diciembre de 2025

  Deceptive UI designs, widely instantiated across the web and commonly known as dark patterns, manipulate users into performing actions misaligned with their goals. In this paper, we show that dark patterns are highly effective in steering agent trajectories, posing a significant risk to agent robust…

• Towards Reliable Evaluation of Adversarial Robustness for Spiking Neural Networks

  Jihang Wang, Dongcheng Zhao, Ruolin Chen, Qian Zhang, Yi Zeng · 30 de diciembre de 2025

  Spiking Neural Networks (SNNs) utilize spike-based activations to mimic the brain's energy-efficient information processing. However, the binary and discontinuous nature of spike activations causes vanishing gradients, making adversarial robustness evaluation via gradient descent unreliable. While i…

• SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents

  Shaofei Cai, Yulei Qin, Haojia Lin, Zihan Xu, Gang Li, Yuchen Shi, Zongyi Li, Yong Mao, Siqi Cai, Xiaoyu Tan, Yitao Liang, Ke Li, Xing Sun · 30 de diciembre de 2025

  Agentic reinforcement learning (RL) holds great promise for the development of autonomous agents under complex GUI tasks, but its scalability remains severely hampered by the verification of task completion. Existing task verification is treated as a passive, post-hoc process: a verifier (i.e., rule…

• Practical challenges of control monitoring in frontier AI deployments

  David Lindner, Charlie Griffin, Tomek Korbak, Roland S. Zimmermann, Geoffrey Irving, Sebastian Farquhar, Alan Cooney · 30 de diciembre de 2025

  Automated control monitors could play an important role in overseeing highly capable AI agents that we do not fully trust. Prior work has explored control monitoring in simplified settings, but scaling monitoring to real-world deployments introduces additional dynamics: parallel agent instances, non…

• ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection

  Zhihao Sun, Haoran Jiang, Haoran Chen, Yixin Cao, Xipeng Qiu, Zuxuan Wu, Yu-Gang Jiang · 30 de diciembre de 2025

  Multimodal large language models have unlocked new possibilities for various multimodal tasks. However, their potential in image manipulation detection remains unexplored. When directly applied to the IMD task, M-LLMs often produce reasoning texts that suffer from hallucinations and overthinking. To…

• Lie to Me: Knowledge Graphs for Robust Hallucination Self-Detection in LLMs

  Sahil Kale, Antonio Luca Alfeo · 30 de diciembre de 2025

  Hallucinations, the generation of apparently convincing yet false statements, remain a major barrier to the safe deployment of LLMs. Building on the strong performance of self-detection methods, we examine the use of structured knowledge representations, namely knowledge graphs, to improve hallucina…

• Scaled-Dot-Product Attention as One-Sided Entropic Optimal Transport

  Elon Litman · 30 de diciembre de 2025

  The scaled-dot-product attention (SDPA) mechanism is a core component of modern deep learning, but its mathematical form is often motivated by heuristics. This work provides a first-principles justification for SDPA. We first show that the attention forward pass is the exact solution to a degenerate…

• Fundamental Novel Consistency Theory: $H$-Consistency Bounds

  Yutao Zhong · 30 de diciembre de 2025

  In machine learning, the loss functions optimized during training often differ from the target loss that defines task performance due to computational intractability or lack of differentiability. We present an in-depth study of the target loss estimation error relative to the surrogate loss estimati…

• Explainable Neural Inverse Kinematics for Obstacle-Aware Robotic Manipulation: A Comparative Analysis of IKNet Variants

  Sheng-Kai Chen, Yi-Ling Tsai, Chun-Chih Chang, Yan-Chen Chen, Po-Chiang Lin · 30 de diciembre de 2025

  Deep neural networks have accelerated inverse-kinematics (IK) inference to the point where low cost manipulators can execute complex trajectories in real time, yet the opaque nature of these models contradicts the transparency and safety requirements emerging in responsible AI regulation. This study…

• Victor Calibration (VC): Multi-Pass Confidence Calibration and CP4.3 Governance Stress Test under Round-Table Orchestration

  Victor Stasiuc · 30 de diciembre de 2025

  Safety alignment can make frontier LMs overly conservative, degrading collaboration via hedging or false refusals. We present a lightweight toolkit with three parts: (1) Victor Calibration (VC), a multi-pass protocol that elicits a scalar confidence proxy T (T0<T1<T2) through iterative evidence re-e…

• Few Tokens Matter: Entropy Guided Attacks on Vision-Language Models

  Mengqi He, Xinyu Tian, Xin Shen, Jinhong Ni, Shu Zou, Zhaoyuan Yang, Jing Zhang · 29 de diciembre de 2025

  Vision-language models (VLMs) achieve remarkable performance but remain vulnerable to adversarial attacks. Entropy, a measure of model uncertainty, is strongly correlated with the reliability of VLM. Prior entropy-based attacks maximize uncertainty at all decoding steps, implicitly assuming that eve…

• Scaling Adversarial Training via Data Selection

  Youran Ye, Dejin Wang, Ajinkya Bhandare · 29 de diciembre de 2025

  Projected Gradient Descent (PGD) is a strong and widely used first-order adversarial attack, yet its computational cost scales poorly, as all training samples undergo identical iterative inner-loop optimization despite contributing unequally to robustness. Motivated by this inefficiency, we propose …

• Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMs

  Jiayu Hu, Beibei Li, Jiangwei Xia, Yanjun Qin, Bing Ji, Zhongshi He · 29 de diciembre de 2025

  While Vision-Language Models (VLMs) have garnered increasing attention in the AI community due to their promising practical applications, they exhibit persistent hallucination issues, generating outputs misaligned with visual inputs. Recent studies attribute these hallucinations to VLMs' over-relian…

• Reflection-Driven Control for Trustworthy Code Agents

  Bin Wang, Jiazheng Quan, Xingrui Yu, Hansen Hu, Yuhao, Ivor Tsang · 29 de diciembre de 2025

  Contemporary large language model (LLM) agents are remarkably capable, but they still lack reliable safety controls and can produce unconstrained, unpredictable, and even actively harmful outputs. To address this, we introduce Reflection-Driven Control, a standardized and pluggable control module th…

• Improving the Convergence Rate of Ray Search Optimization for Query-Efficient Hard-Label Attacks

  Xinjie Xu, Shuyu Cheng, Dongwei Xu, Qi Xuan, Chen Ma · 25 de diciembre de 2025

  In hard-label black-box adversarial attacks, where only the top-1 predicted label is accessible, the prohibitive query complexity poses a major obstacle to practical deployment. In this paper, we focus on optimizing a representative class of attacks that search for the optimal ray direction yielding…

• Time-Efficient Evaluation and Enhancement of Adversarial Robustness in Deep Neural Networks

  Runqi Lin · 25 de diciembre de 2025

  With deep neural networks (DNNs) increasingly embedded in modern society, ensuring their safety has become a critical and urgent issue. In response, substantial efforts have been dedicated to the red-blue adversarial framework, where the red team focuses on identifying vulnerabilities in DNNs and th…

• Robustness Certificates for Neural Networks against Adversarial Attacks

  Sara Taheri, Mahalakshmi Sabanayagam, Debarghya Ghoshdastidar, Majid Zamani · 25 de diciembre de 2025

  The increasing use of machine learning in safety-critical domains amplifies the risk of adversarial threats, especially data poisoning attacks that corrupt training data to degrade performance or induce unsafe behavior. Most existing defenses lack formal guarantees or rely on restrictive assumptions…

• Safety Alignment of LMs via Non-cooperative Games

  Anselm Paulus, Ilia Kulikov, Brandon Amos, R\'emi Munos, Ivan Evtimov, Kamalika Chaudhuri, Arman Zharmagambetov · 25 de diciembre de 2025

  Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment. Current approaches rely on sequential adversarial training: generating adversarial prompts and fine-tuning LMs to defend against them. We introduce a different paradigm: fram…

• Beyond Context: Large Language Models Failure to Grasp Users Intent

  Ahmed M. Hussain, Salahuddin Salahuddin, Panos Papadimitratos · 25 de diciembre de 2025

  Current Large Language Models (LLMs) safety approaches focus on explicitly harmful content while overlooking a critical vulnerability: the inability to understand context and recognize user intent. This creates exploitable vulnerabilities that malicious users can systematically leverage to circumven…

• MicroProbe: Efficient Reliability Assessment for Foundation Models with Minimal Data

  Aayam Bansal, Ishaan Gangwani · 25 de diciembre de 2025

  Foundation model reliability assessment typically requires thousands of evaluation examples, making it computationally expensive and time-consuming for real-world deployment. We introduce microprobe, a novel approach that achieves comprehensive reliability assessment using only 100 strategically sel…

• RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic

  Le Wang, Zonghao Ying, Xiao Yang, Quanchen Zou, Zhenfei Yin, Tianlin Li, Jian Yang, Yaodong Yang, Aishan Liu, Xianglong Liu · 25 de diciembre de 2025

  Embodied agents powered by vision-language models (VLMs) are increasingly capable of executing complex real-world tasks, yet they remain vulnerable to hazardous instructions that may trigger unsafe behaviors. Runtime safety guardrails, which intercept hazardous actions during task execution, offer a…

• Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits

  Yizhak Yisrael Elboher, Avraham Raviv, Amihay Elboher, Zhouxing Shi, Omri Azencot, Hillel Kugler, Guy Katz · 25 de diciembre de 2025

  Ensuring the safety and efficiency of AI systems is a central goal of modern research. Formal verification provides guarantees of neural network robustness, while early exits improve inference efficiency by enabling intermediate predictions. Yet verifying networks with early exits introduces new cha…

• Defending against adversarial attacks using mixture of experts

  Mohammad Meymani, Roozbeh Razavi-Far · 25 de diciembre de 2025

  Machine learning is a powerful tool enabling full automation of a huge number of tasks without explicit programming. Despite recent progress of machine learning in different domains, these models have shown vulnerabilities when they are exposed to adversarial threats. Adversarial threats aim to hind…