Speech Recognition and Synthesis

Latest papers

• GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model

  Haoyang Li, Xuyi Zhuang, Azmat Adnan, Ye Ni, Wei Rao, Shreyas Gopal, Eng Siong Chng · 25 December 2025

  Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech. We present GenTSE, a two-stage decoder-only generative LM approach for TSE: Stage-1 predicts coarse semantic tokens, and Stage-2 genera…

• SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation

  Mahi Luthra, Jiayi Shen, Maxime Poli, Angelo Ortiz, Yosuke Higuchi, Youssef Benchekroun, Martin Gleize, Charles-Eric Saint-James, Dongyan Lin, Phillip Rust, Angel Villar, Surya Parimi, Vanessa Stark, Rashel Moritz, Juan Pino, Yann LeCun, Emmanuel Dupoux · 25 December 2025

  Human infants, with only a few hundred hours of speech exposure, acquire basic units of new languages, highlighting a striking efficiency gap compared to the data-hungry self-supervised speech models. To address this gap, this paper introduces SpidR-Adapt for rapid adaptation to new languages using …

• Fun-Audio-Chat Technical Report

  Qian Chen, Luyao Cheng, Chong Deng, Xiangang Li, Jiaqing Liu, Chao-Hong Tan, Wen Wang, Junhao Xu, Jieping Ye, Qinglin Zhang, Qiquan Zhang, Jingren Zhou · 24 December 2025

  Recent advancements in joint speech-text models show great potential for seamless voice interactions. However, existing models face critical challenges: temporal resolution mismatch between speech tokens (25Hz) and text tokens (~3Hz) dilutes semantic information, incurs high computational costs, and…

• TICL+: A Case Study On Speech In-Context Learning for Children's Speech Recognition

  Haolong Zheng, Yekaterina Yegorova, Mark Hasegawa-Johnson · 23 December 2025

  Children's speech recognition remains challenging due to substantial acoustic and linguistic variability, limited labeled data, and significant differences from adult speech. Speech foundation models can address these challenges through Speech In-Context Learning (SICL), allowing adaptation to new d…

• Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis

  Pengchao Feng, Yao Xiao, Ziyang Ma, Zhikang Niu, Shuai Fan, Yao Li, Sheng Wang, Xie Chen · 23 December 2025

  Recent advances in text-to-speech (TTS) have yielded remarkable improvements in naturalness and intelligibility. Building on these achievements, research has increasingly shifted toward enhancing the expressiveness of generated speech, such as dialectal and emotional TTS. However, cross-style synthe…

• Supplementary Resources and Analysis for Automatic Speech Recognition Systems Trained on the Loquacious Dataset

  Nick Rossenbach, Robin Schmitt, Tina Raissi, Simon Berger, Larissa Kleppel, Ralf Schl\"uter · 23 December 2025

  The recently published Loquacious dataset aims to be a replacement for established English automatic speech recognition (ASR) datasets such as LibriSpeech or TED-Lium. The main goal of the Loquacious dataset is to provide properly defined training and test partitions across many acoustic and languag…

• ASR-Synchronized Speaker-Role Diarization

  Arindam Ghosh, Mark Fuhs, Bongjun Kim, Anurag Chowdhury, Monika Woszczyna · 23 December 2025

  Speaker-role diarization (RD), such as doctor vs. patient or lawyer vs. client, is practically often more useful than conventional speaker diarization (SD), which assigns only generic labels (speaker-1, speaker-2). The state-of-the-art end-to-end ASR+RD approach uses a single transducer that seriali…

• Fun-ASR Technical Report

  Keyu An, Yanni Chen, Zhigao Chen, Chong Deng, Zhihao Du, Changfeng Gao, Zhifu Gao, Bo Gong, Xiangang Li, Yabin Li, Ying Liu, Xiang Lv, Yunjie Ji, Yiheng Jiang, Bin Ma, Haoneng Luo, Chongjia Ni, Zexu Pan, Yiping Peng, Zhendong Peng, Peiyao Wang, Hao Wang, Haoxu Wang, Wen Wang, Wupeng Wang, Yuzhong Wu, Biao Tian, Zhentao Tan, Nan Yang, Bin Yuan, Jieping Ye, Jixing Yu, Qinglin Zhang, Kun Zou, Han Zhao, Shengkui Zhao, Jingren Zhou, Yanqiao Zhu · 22 December 2025

  In recent years, automatic speech recognition (ASR) has witnessed transformative advancements driven by three complementary paradigms: data scaling, model size scaling, and deep integration with large language models (LLMs). However, LLMs are prone to hallucination, which can significantly degrade u…

• Incorporating Error Level Noise Embedding for Improving LLM-Assisted Robustness in Persian Speech Recognition

  Zahra Rahmani (Department of Computer Engineering, Sharif University of Technology), Hossein Sameti (Department of Computer Engineering, Sharif University of Technology) · 22 December 2025

  Automatic Speech Recognition (ASR) systems suffer significant performance degradation in noisy environments, a challenge that is especially severe for low-resource languages such as Persian. Even state-of-the-art models such as Whisper struggle to maintain accuracy under varying signal-to-noise rati…

• Robust TTS Training via Self-Purifying Flow Matching for the WildSpoof 2026 TTS Track

  June Young Yi, Hyeongju Kim, Juheon Lee · 22 December 2025

  This paper presents a lightweight text-to-speech (TTS) system developed for the WildSpoof Challenge TTS Track. Our approach fine-tunes the recently released open-weight TTS model, \textit{Supertonic}\footnote{\url{https://github.com/supertone-inc/supertonic}}, with Self-Purifying Flow Matching (SPFM…

• When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems

  Sujal Chondhekar, Vasanth Murukuri, Rushabh Vasani, Sanika Goyal, Rajshree Badami, Anushree Rana, Sanjana SN, Karthik Pandia, Sulabh Katiyar, Neha Jagadeesh, Sankalp Gulati · 22 December 2025

  Speech enhancement methods are commonly believed to improve the performance of automatic speech recognition (ASR) in noisy environments. However, the effectiveness of these techniques cannot be taken for granted in the case of modern large-scale ASR models trained on diverse, noisy data. We present …

• Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs

  Sara Papi, Javier Garcia Gilabert, Zachary Hopton, Vil\'em Zouhar, Carlos Escolano, Gerard I. G\'allego, Jorge Iranzo-S\'anchez, Ahrii Kim, Dominik Mach\'a\v{c}ek, Patricia Schmidtova, Maike Z\"ufle · 19 December 2025

  As Large Language Models (LLMs) expand beyond text, integrating speech as a native modality has given rise to SpeechLLMs, which aim to translate spoken language directly, thereby bypassing traditional transcription-based pipelines. Whether this integration improves speech-to-text translation quality…

• Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalization

  Tzu-Quan Lin, Wei-Ping Huang, Hao Tang, Hung-yi Lee · 19 December 2025

  Fine-tuning speech representation models can enhance performance on specific tasks but often compromises their cross-task generalization ability. This degradation is often caused by excessive changes in the representations, making it difficult to retain information learned during pre-training. Exist…

• LLaDA2.0: Scaling Up Diffusion Language Models to 100B

  Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, Chengxi Li, Chongxuan Li, Jianguo Li, Zehuan Li, Huabin Liu, Ling Liu, Guoshan Lu, Xiaocheng Lu, Yuxin Ma, Jianfeng Tan, Lanning Wei, Ji-Rong Wen, Yipeng Xing, Xiaolu Zhang, Junbo Zhao, Da Zheng, Jun Zhou, Junlin Zhou, Zhanchao Zhou, Liwang Zhu, Yihong Zhuang · 19 December 2025

  This paper presents LLaDA2.0 -- a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (AR) models -- establishing a new paradigm for frontier-scale deployment. Instead of costly training from scratch, LLaDA2.…

• Pseudo-Cepstrum: Pitch Modification for Mel-Based Neural Vocoders

  Nikolaos Ellinas, Alexandra Vioni, Panos Kakoulidis, Georgios Vamvoukakis, Myrsini Christidou, Konstantinos Markopoulos, Junkwang Oh, Gunu Jho, Inchul Hwang, Aimilios Chalamandaris, Pirros Tsiakoulis · 19 December 2025

  This paper introduces a cepstrum-based pitch modification method that can be applied to any mel-spectrogram representation. As a result, this method is compatible with any mel-based vocoder without requiring any additional training or changes to the model. This is achieved by directly modifying the …

• O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization

  Elio Gruttadauria (IP Paris, LTCI, IDS, S2A), Mathieu Fontaine (LTCI, IP Paris), Jonathan Le Roux (IDS, S2A, LTCI), Slim Essid (IDS, S2A, LTCI) · 18 December 2025

  We introduce O-EENC-SD: an end-to-end online speaker diarization system based on EEND-EDA, featuring a novel RNN-based stitching mechanism for online prediction. In particular, we develop a novel centroid refinement decoder whose usefulness is assessed through a rigorous ablation study. Our system p…

• Joint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spotting

  Ramesh Gundluru, Shubham Gupta, Sri Rama Murty K · 17 December 2025

  Acoustic Word Embeddings (AWEs) improve the efficiency of speech retrieval tasks such as Spoken Term Detection (STD) and Keyword Spotting (KWS). However, existing approaches suffer from limitations, including unimodal supervision, disjoint optimization of audio-audio and audio-text alignment, and th…

• Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximation

  Saierdaer Yusuyin, Te Ma, Hao Huang, Zhijian Ou · 17 December 2025

  Recently, pre-trained models with phonetic supervision have demonstrated their advantages for crosslingual speech recognition in data efficiency and information sharing across languages. However, a limitation is that a pronunciation lexicon is needed for such phoneme-based crosslingual speech recogn…

• Protecting Bystander Privacy via Selective Hearing in Audio LLMs

  Xiao Zhan, Guangzhi Sun, Jose Such, Phil Woodland · 16 December 2025

  Audio Large language models (LLMs) are increasingly deployed in the real world, where they inevitably capture speech from unintended nearby bystanders, raising privacy risks that existing benchmarks and defences did not consider. We introduce SH-Bench, the first benchmark designed to evaluate select…

• Layer-aware TDNN: Speaker Recognition Using Multi-Layer Features from Pre-Trained Models

  Jin Sob Kim, Hyun Joon Park, Wooseok Shin, Juan Yun, Sung Won Han · 16 December 2025

  Recent advances in self-supervised learning (SSL) on Transformers have significantly improved speaker verification (SV) by providing domain-general speech representations. However, existing approaches have underutilized the multi-layered nature of SSL encoders. To address this limitation, we propose…

• Lightweight Model Attribution and Detection of Synthetic Speech via Audio Residual Fingerprints

  Mat\'ias Pizarro, Mike Laszkiewicz, Dorothea Kolossa, Asja Fischer · 12 December 2025

  As speech generation technologies advance, so do risks of impersonation, misinformation, and spoofing. We present a lightweight, training-free approach for detecting synthetic speech and attributing it to its source model. Our method addresses three tasks: (1) single-model attribution in an open-wor…

• Towards Robust Assessment of Pathological Voices via Combined Low-Level Descriptors and Foundation Model Representations

  Whenty Ariyanti, Kuan-Yu Chen, Sabato Marco Siniscalchi, Hsin-Min Wang, Yu Tsao · 12 December 2025

  Perceptual voice quality assessment plays a vital role in diagnosing and monitoring voice disorders. Traditional methods, such as the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) and the Grade, Roughness, Breathiness, Asthenia, and Strain (GRBAS) scales, rely on expert raters and are p…

• A Minimalist Optimizer Design for LLM Pretraining

  Athanasios Glentis, Jiaxiang Li, Andi Han, Mingyi Hong · 11 December 2025

  Training large language models (LLMs) typically relies on adaptive optimizers such as Adam, which introduce extra operations and require significant more memory to maintain first- and second-order moments than SGD. While recent works such as GaLore, Fira and APOLLO have proposed state-compressed var…

• Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation

  Xueyao Zhang, Junan Zhang, Yuancheng Wang, Chaoren Wang, Yuanzhe Chen, Dongya Jia, Zhuo Chen, Zhizheng Wu · 11 December 2025

  Controllable human voice generation, particularly for expressive domains like singing, remains a significant challenge. This paper introduces Vevo2, a unified framework for controllable speech and singing voice generation. To tackle issues like the scarcity of annotated singing data and to enable fl…

• Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual Speech Recognition Evaluation

  Vaibhav Srivastav, Steven Zheng, Eric Bezzam, Eustache Le Bihan, Adel Moumen, Sanchit Gandhi · 11 December 2025

  Despite rapid progress, ASR evaluation remains saturated with short-form English, and efficiency is rarely reported. We present the Open ASR Leaderboard, a fully reproducible benchmark and interactive leaderboard comparing 60+ open-source and proprietary systems across 11 datasets, including a dedic…