Adversarial Robustness in Machine Learning

Últimos artículos

• Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models

  Ye Yu, Haibo Jin, Yaoning Yu, Jun Zhuang, Haohan Wang · 2 de febrero de 2026

  Large audio-language models increasingly operate on raw speech inputs, enabling more seamless integration across domains such as voice assistants, education, and clinical triage. This transition, however, introduces a distinct class of vulnerabilities that remain largely uncharacterized. We examine …

• THINKSAFE: Self-Generated Safety Alignment for Reasoning Models

  Seanie Lee, Sangwoo Park, Yumin Choi, Gyeongman Kim, Minki Kang, Jihun Yun, Dongmin Park, Jongho Park, Sung Ju Hwang · 2 de febrero de 2026

  Large reasoning models (LRMs) achieve remarkable performance by leveraging reinforcement learning (RL) on reasoning tasks to generate long chain-of-thought (CoT) reasoning. However, this over-optimization often prioritizes compliance, making models vulnerable to harmful prompts. To mitigate this saf…

• Stealthy Poisoning Attacks Bypass Defenses in Regression Settings

  Javier Carnerero-Cano, Luis Mu\~noz-Gonz\'alez, Phillippa Spencer, Emil C. Lupu · 2 de febrero de 2026

  Regression models are widely used in industrial processes, engineering and in natural and physical sciences, yet their robustness to poisoning has received less attention. When it has, studies often assume unrealistic threat models and are thus less useful in practice. In this paper, we propose a no…

• From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models

  Farnaz Soltaniani, Mohammad Ghafari · 2 de febrero de 2026

  Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly similar samples. When such samples are split across training and test sets, data leakage may occur, …

• Exact closed-form Gaussian moments of residual layers

  Simon Kuang, Xinfan Lin · 2 de febrero de 2026

  We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment matching. We close a longstanding gap by deriving exact moment matching for the probit, GeLU, ReLU (as a limit of GeLU), He…

• SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks

  Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri · 2 de febrero de 2026

  Early-exit neural networks have become popular for reducing inference latency by allowing intermediate predictions when sufficient confidence is achieved. However, standard approaches typically rely on single-model confidence thresholds, which are frequently unreliable due to inherent calibration is…

• Quantifying Model Uniqueness in Heterogeneous AI Ecosystems

  Lei You · 2 de febrero de 2026

  As AI systems evolve from isolated predictors into complex, heterogeneous ecosystems of foundation models and specialized adapters, distinguishing genuine behavioral novelty from functional redundancy becomes a critical governance challenge. Here, we introduce a statistical framework for auditing mo…

• Make Anything Match Your Target: Universal Adversarial Perturbations against Closed-Source MLLMs via Multi-Crop Routed Meta Optimization

  Hui Lu, Yi Yu, Yiming Yang, Chenyu Yi, Xueyi Ke, Qixing Zhang, Bingquan Shen, Alex Kot, Xudong Jiang · 2 de febrero de 2026

  Targeted adversarial attacks on closed-source multimodal large language models (MLLMs) have been increasingly explored under black-box transfer, yet prior methods are predominantly sample-specific and offer limited reusability across inputs. We instead study a more stringent setting, Universal Targe…

• FNF: Functional Network Fingerprint for Large Language Models

  Yiheng Liu, Junhao Ning, Sichen Xia, Haiyang Sun, Yang Yang, Hanyang Chi, Xiaohui Gao, Ning Qiang, Bao Ge, Junwei Han, Xintao Hu · 2 de febrero de 2026

  The development of large language models (LLMs) is costly and has significant commercial value. Consequently, preventing unauthorized appropriation of open-source LLMs and protecting developers' intellectual property rights have become critical challenges. In this work, we propose the Functional Net…

• A Real-Time Privacy-Preserving Behavior Recognition System via Edge-Cloud Collaboration

  Huan Song, Shuyu Tian, Junyi Hao, Cheng Yuan, Zhenyu Jia, Jiawei Shao, Xuelong Li · 2 de febrero de 2026

  As intelligent sensing expands into high-privacy environments such as restrooms and changing rooms, the field faces a critical privacy-security paradox. Traditional RGB surveillance raises significant concerns regarding visual recording and storage, while existing privacy-preserving methods-ranging …

• AI Kill Switch for malicious web-based LLM agent

  Sechan Lee, Sangdon Park · 2 de febrero de 2026

  Recently, web-based Large Language Model (LLM) agents autonomously perform increasingly complex tasks, thereby bringing significant convenience. However, they also amplify the risks of malicious misuse cases such as unauthorized collection of personally identifiable information (PII), generation of …

• Multi-Level Safety Continual Projection for Fine-Tuned Large Language Models without Retraining

  Bing Han, Feifei Zhao, Dongcheng Zhao, Guobin Shen, Ping Wu, Yu Shi, Yi Zeng · 2 de febrero de 2026

  While fine-tuning services drive the rapid expansion of task capabilities in large language models (LLMs), they are often accompanied by the degradation and reorganization of safety-aligned representations, making models more prone to deviating from human preferences and exposing them to emerging ja…

• Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment

  Yavuz Bakman, Duygu Nur Yaldiz, Salman Avestimehr, Sai Praneeth Karimireddy · 2 de febrero de 2026

  Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed "aligned" can exhibit misaligned behavior after fine-tuning, such as forgetting jailbreak safety features or re-surfacing knowledge that …

• PoSafeNet: Safe Learning with Poset-Structured Neural Nets

  Kiwan Wong, Wei Xiao, Daniela Rus · 2 de febrero de 2026

  Safe learning is essential for deploying learningbased controllers in safety-critical robotic systems, yet existing approaches often enforce multiple safety constraints uniformly or via fixed priority orders, leading to infeasibility and brittle behavior. In practice, safety requirements are heterog…

• Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling

  Mingqian Feng, Xiaodong Liu, Weiwei Yang, Chenliang Xu, Christopher White, Jianfeng Gao · 2 de febrero de 2026

  Large Language Models (LLMs) are typically evaluated for safety under single-shot or low-budget adversarial prompting, which underestimates real-world risk. In practice, attackers can exploit large-scale parallel sampling to repeatedly probe a model until a harmful response is produced. While recent…

• UCPO: Uncertainty-Aware Policy Optimization

  Xianzhou Zeng, Jing Huang, Chunmei Xie, Gongrui Nan, Siye Chen, Mengyu Lu, Weiqi Xiong, Qixuan Zhou, Junhao Zhang, Qiang Zhu, Yadong Li, Xingzhong Xu · 2 de febrero de 2026

  The key to building trustworthy Large Language Models (LLMs) lies in endowing them with inherent uncertainty expression capabilities to mitigate the hallucinations that restrict their high-stakes applications. However, existing RL paradigms such as GRPO often suffer from Advantage Bias due to binary…

• Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory

  Yuhao Zhan, Tianyu Fan, Linxuan Huang, Zirui Guo, Chao Huang · 2 de febrero de 2026

  Diagnosing the failure mechanisms of Deep Research Agents (DRAs) remains a critical challenge. Existing benchmarks predominantly rely on end-to-end evaluation, obscuring critical intermediate hallucinations, such as flawed planning, that accumulate throughout the research trajectory. To bridge this …

• A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy

  Pedro H. Barcha Correia, Ryan W. Achjian, Diego E. G. Caetano de Oliveira, Ygor Acacio Maria, Victor Takashi Hayashi, Marcos Lopes, Charles Christian Miers, Marcos A. Simplicio Jr · 2 de febrero de 2026

  The rapid advancement and widespread adoption of generative artificial intelligence (GenAI) and large language models (LLMs) has been accompanied by the emergence of new security vulnerabilities and challenges, such as jailbreaking and other prompt injection attacks. These maliciously crafted inputs…

• The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples

  Hsiang Hsu, Pradeep Niroula, Zichang He, Ivan Brugere, Freddy Lecue, Chun-Fu Chen · 2 de febrero de 2026

  Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model ou…

• Jailbreaks on Vision Language Model via Multimodal Reasoning

  Aarush Noheria, Yuguang Yao · 2 de febrero de 2026

  Vision-language models (VLMs) have become central to tasks such as visual question answering, image captioning, and text-to-image generation. However, their outputs are highly sensitive to prompt variations, which can reveal vulnerabilities in safety alignment. In this work, we present a jailbreak f…

• Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models

  Charles Westphal, Keivan Navaie, Fernando E. Rosas · 2 de febrero de 2026

  Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In respo…

• Protecting Private Code in IDE Autocomplete using Differential Privacy

  Evgeny Grigorenko, David Stanojevi\'c, David Ili\'c, Egor Bogomolov, Kostadin Cvejoski · 2 de febrero de 2026

  Modern Integrated Development Environments (IDEs) increasingly leverage Large Language Models (LLMs) to provide advanced features like code autocomplete. While powerful, training these models on user-written code introduces significant privacy risks, making the models themselves a new type of data v…

• Language Models That Walk the Talk: A Framework for Formal Fairness Certificates

  Danqing Chen, Tobias Ladner, Ahmed Rayen Mhadhbi, Matthias Althoff · 2 de febrero de 2026

  As large language models become integral to high-stakes applications, ensuring their robustness and fairness is critical. Despite their success, large language models remain vulnerable to adversarial attacks, where small perturbations, such as synonym substitutions, can alter model predictions, posi…

• InvThink: Towards AI Safety via Inverse Reasoning

  Yubin Kim, Taehan Kim, Eugene Park, Chunjong Park, Cynthia Breazeal, Daniel McDuff, Hae Won Park · 2 de febrero de 2026

  We present InvThink, a simple yet powerful approach that gives language models the capability of inverse thinking: reasoning through failure modes before generating responses. Unlike existing safety alignment methods that optimize directly for safe response, InvThink instructs models to 1) enumerate…

• How should AI Safety Benchmarks Benchmark Safety?

  Cheng Yu, Severin Engelmann, Ruoxuan Cao, Dalia Ali, Orestis Papakyriakopoulos · 2 de febrero de 2026

  AI safety benchmarks are pivotal for safety in advanced AI systems; however, they have significant technical, epistemic, and sociotechnical shortcomings. We present a review of 210 safety benchmarks that maps out common challenges in safety benchmarking, documenting failures and limitations by drawi…