Domain Adaptation and Few-Shot Learning

Neueste Paper

• Task-Oriented Rank Adaptation for Continual Learning in Text Classification

  Rey Sanchez Lopez, Eduardo Morales Manzanares, Hugo Jair Escalante · 2. Oktober 2026

  Continual learning (CL) in text classification faces two critical challenges: catastrophic forgetting and negative transfer across sequential tasks. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA enable efficient adaptation by learning low-rank updates of the model parameters. However, …

• Local Support Learning

  Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes · 2. Oktober 2026

  We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of each weight matrix, we uncover a natural retention objective under which updates produced by gradient-based optimizers are suboptimal. Following this …

• Capturing In-Context Learning Dynamics with Task Operators

  Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang · 2. Oktober 2026

  In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work co…

• Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweighting

  Hongyu Cao, Xinyuan Wang, Arun Vignesh Malarkkan, Kunpeng Liu, Haifeng Chen, Yanjie Fu · 2. Oktober 2026

  In many industrial applications, 1) tabular data is scarce and imbalanced and thus requires synthetic expansion; 2) input distributions drift between training and deployment (covariate shift); 3) validation sets often diverge from unseen test environments; or 4) standard generative models simply mim…

• On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D

  Fabio J. Fehr, Philip Torr · 2. Oktober 2026

  General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires task-specific training or adaptation. We ask whether they can instead be generated directly from a few demonstrations. Us…

• Restoring without Forgetting: Filter-Level Continual Image Restoration via Parameter-Space Integrated Gradients

  Xin Feng, Jin Zhao, Yizhen Zhang, Wenjie Pei, Fanglin Chen, Guangming Lu · 1. Oktober 2026

  Adapting image restoration models to a stream of new tasks without revisiting past data remains challenging due to catastrophic forgetting. In this work, we propose Restoring without Forgetting (RwF), a filter-level continual adaptation framework for image restoration built upon a critical observati…

• Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging

  Zijing Wang, Yongkang Liu, Mingyang Wang, Ercong Nie, Mengjie Zhao, Yunpu Ma, Kang Liu, Zihan Wang, Shi Feng, Daling Wang, Hinrich Sch\"utze · 1. Oktober 2026

  Merging pretrained models has emerged as an effective approach for consolidating diverse capabilities into a single unified model. However, prevailing merging methods typically treat each task vector as an indivisible merging unit, overlooking the heterogeneous geometric changes encoded within it. T…

• Unmerge: Efficient Machine Unlearning via Task Arithmetic

  Haoran Tang, Andrew Tan, Rajiv Khanna · 1. Oktober 2026

  Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlear…

• Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention

  Luis Zuin, Alexis Huet, Dario Rossi, Zied Ben Houidi · 1. Oktober 2026

  Self-distillation with privileged context adapts a language model from demonstrations by letting the model, once conditioned on a reference response, teach its context-free copy token by token. Our taxonomy reveals existing methods differ along three entangled axes: (i) the rollout source (student o…

• Learning What to Forget: Distributional Unlearning for LLM Representation Spaces

  Pinaki Mohanty, Haoran Tang, Maggie Makar, Rajiv Khanna · 1. Oktober 2026

  Machine learning systems increasingly face the need to remove the influence of entire data domains, such as toxic language, harmful behavior, or topical content, rather than isolated records. Recent work formalizes this problem as \emph{distributional unlearning}: selecting a subset of a forget doma…

• Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features

  Cunchun Li, Haonan He, Yifan Gao, Minglei Li, Jingqi Ye, Qingyu Yang, Peng Ye · 30. September 2026

  Supervised fine-tuning (SFT) learns most aggressively from tokens that the model deems least likely. This helps acquire new behaviors, but also amplifies noisy or conflicting supervision and can overwrite useful pretrained knowledge. Through a unified policy-loss view, we revisit existing token-rewe…

• Width Expansion as a Method for Class Incremental Learning

  A. L. S. Conde, Y. Elkhatib, C. M. Ranieri · 30. September 2026

  Class Incremental Learning (Class-IL) requires models to learn new classes over time while preserving previously acquired knowledge without access to past data or task identity. This setting intensifies the stability-plasticity dilemma and makes catastrophic forgetting a central challenge. Existing …

• Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection

  Yuankun Xie, Xiaoxuan Guo, Xiaopeng Wang, Siqing Qin, Shaole Li, Kong Aik Lee · 30. September 2026

  Continual audio deepfake detection requires learning newly emerging deepfake methods while retaining discrimination of previously encountered speech. Existing dataset-incremental evaluation changes both real-speech domains and deepfake mechanisms, making their effects difficult to distinguish. We co…

• Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning

  Chiyuan He, Zihuan Qiu, Fanman Meng, Chao Wang, Liangjiang Chen, Linfeng Xu, Qingbo Wu, Hongliang Li · 30. September 2026

  Benefiting from transferable visual-textual alignment, CLIP has been widely adopted for class-incremental learning (CIL). However, existing learners either repeatedly update components shared across tasks, leading to knowledge overwriting, or overly isolate new-task updates, hindering the reuse of C…

• HiTS-CL: A Continual Learning Framework for Long-Horizon Temporal Knowledge Graph Extrapolation

  Yansong Liu, Rui Liu, Yuan Zuo, Hongwei Zhao, Da Fu, Fuwei Zhang, Fuzhen Zhuang, Yong Chen, Zhe Li · 30. September 2026

  Extrapolative temporal knowledge graph reasoning (TKGR) predicts future facts from historical snapshots. Most existing methods train once on an early prefix of the timeline and then use a frozen model for all future timestamps. We argue that this fixed-prefix protocol is misaligned with extrapolatio…

• Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time

  Jae-Ho Lee, Min-Yeong Park, Jun-Yeong Moon, Jung Uk Kim, Gyeong-Moon Park · 30. September 2026

  Continual learning enables vision systems to adapt to ever-changing data distributions. Despite significant advances, existing approaches fail to capture continuous and concurrent shifts in classes and domains, a critical capability for real-world deployment. This work introduces Online VIL (Online …

• Longer Records, Broader Invariance: The Hidden Scaling Problem in Longitudinal Contrastive Learning

  Rameen Mahmood, Xuhai "Orson" Xu, Zachary Beattie, Jeffrey Kaye, Danny Yuxing Huang · 30. September 2026

  Longitudinal data are valuable because people change. Yet the objectives used to learn from these data can inadvertently erase that change. In person-level contrastive learning, observations from the same person are treated as positives; as records grow, those positives can span increasingly distant…

• AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

  Konstantinos D. Polyzos, Eleni Oikonomou, Tara Javidi · 30. September 2026

  Large foundation models have been introduced with the promise of efficient adaptation to downstream tasks. Yet, under limited supervision, MLLMs, an important class of large foundation models, remain challenging to adapt to various downstream tasks. Adaptation typically relies either on MLLM paramet…

• Normative Loss Landscape Navigation: A Trajectory-Based Approach to Mitigating Forgetting in Incremental Learning

  Isabelle Aguilar, Zayn Andre Zainal, Luis Fernando Herbozo Contreras, Zhaojing Huang, Omid Kavehei · 30. September 2026

  Continual learning models suffer from catastrophic forgetting when trained sequentially on non-stationary data distributions. Previously, this has been addressed through weight regularization. While preconditioning gradients offer a promising alternative to mitigate forgetting, current approaches ar…

• WISE-ATTA: When to Ask for Labels in Budgeted Active Test-Time Adaptation

  Muhammad Huzaifa, Lea Sch\"onherr, Thorsten Eisenhofer · 30. September 2026

  Active test-time adaptation (ATTA) improves robustness under distribution shift by updating a deployed model during inference while selectively querying supervision. However, most existing ATTA methods implicitly assume that supervision can be requested for every incoming test batch, which can incur…

• Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition

  Yihao Ouyang, Shiwei Li, Haozhao Wang, Xiandi Luo, Zhuoqi Hu, Jinglun Yu, Yichen Li, Ruixuan Li · 30. September 2026

  Low-Rank Adaptation (LoRA) is a widely used approach to parameter-efficient fine-tuning (PEFT), yet a performance gap can remain relative to full fine-tuning (FFT). Many LoRA variants improve the initialization or optimization of low-rank factors. At each training step, however, their first-order we…

• GradLev: Token-Parallel Test-Time Training Via Costate Prediction

  Bo Liu, Qiang Liu · 29. September 2026

  Test-time training (TTT) allows a model to improve its predictions at inference time by updating weights after every observed token. However, sequential gra- dient writes make parallel training difficult. We observe that, given layer inputs and activation gradients (costates), online gradient descen…

• WhiteCon: Semi-Supervised Domain Adaptation Regression Through Whitening Transform and Dual Consistency

  Se Jin Sim, Seoung Bum Kim · 29. September 2026

  Domain adaptation is crucial for addressing distributional shifts that degrade model performance across domains. While most existing research has centered on classification, semi-supervised domain adaptation regression (SSDAR) for continuous-output tasks remains largely unexplored, particularly in p…

• Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

  Yi Ren, Wenlong Deng, Guanzhe Hong, Clare Lyle, Yarin Gal · 29. September 2026

  Modern language models are likely to be updated throughout their lifetime rather than trained once and frozen. Each update therefore participates in a recurring cycle: decide which experience to learn from, understand what that update changes, and remain capable of learning from what comes next. We …

• Continual Learning via Self-Probe Gradients

  Dongkyu Cho, Rumi Chunara, Sungmin Cha · 29. September 2026

  Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that language models can expand this evidence through self-pro…