Speech Recognition and Synthesis

Neueste Paper

• Training-Free Intelligibility-Guided Observation Addition for Noisy ASR

  Haoyang Li, Changsong Liu, Wei Rao, Hao Shi, Sakriani Sakti, Eng Siong Chng · 25. Februar 2026

  Automatic speech recognition (ASR) degrades severely in noisy environments. Although speech enhancement (SE) front-ends effectively suppress background noise, they often introduce artifacts that harm recognition. Observation addition (OA) addressed this issue by fusing noisy and SE enhanced speech, …

• Assessing the Impact of Speaker Identity in Speech Spoofing Detection

  Anh-Tuan Dao, Driss Matrouf, Nicholas Evans · 25. Februar 2026

  Spoofing detection systems are typically trained using diverse recordings from multiple speakers, often assuming that the resulting embeddings are independent of speaker identity. However, this assumption remains unverified. In this paper, we investigate the impact of speaker information on spoofing…

• StyleStream: Real-Time Zero-Shot Voice Style Conversion

  Yisi Liu, Nicholas Lee, Gopala Anumanchipalli · 24. Februar 2026

  Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voic…

• CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Data

  Qibing Bai, Shuhao Shi, Shuai Wang, Yukai Ju, Yannan Wang, Haizhou Li · 24. Februar 2026

  Accent normalization (AN) systems often struggle with unnatural outputs and undesired content distortion, stemming from both suboptimal training data and rigid duration modeling. In this paper, we propose a "source-synthesis" methodology for training data construction. By generating source L2 speech…

• CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment

  Hanwen Liu, Saierdaer Yusuyin, Hao Huang, Zhijian Ou · 24. Februar 2026

  Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis qualit…

• MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows

  Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, Yuto Kondo · 23. Februar 2026

  In voice conversion (VC) applications, diffusion and flow-matching models have exhibited exceptional speech quality and speaker similarity performances. However, they are limited by slow conversion owing to their iterative inference. Consequently, we propose MeanVoiceFlow, a novel one-step nonparall…

• Speech to Speech Synthesis for Voice Impersonation

  Bjorn Johnson, Jared Levy · 20. Februar 2026

  Numerous models have shown great success in the fields of speech recognition as well as speech synthesis, but models for speech to speech processing have not been heavily explored. We propose Speech to Speech Synthesis Network (STSSN), a model based on current state of the art systems that fuses the…

• Discrete optimal transport is a strong audio adversarial attack

  Anton Selitskiy, Akib Shahriyar, Jishnuraj Prakasan · 20. Februar 2026

  In this paper, we introduce the discrete optimal transport voice conversion ($k$DOT-VC) method. Comparison with $k$NN-VC, SinkVC, and Gaussian optimal transport (MKL) demonstrates stronger domain adaptation abilities of our method. We use the probabilistic nature of optimal transport (OT) and show t…

• The Cascade Equivalence Hypothesis: When Do Speech LLMs Behave Like ASR$\rightarrow$LLM Pipelines?

  Jayadev Billa · 20. Februar 2026

  Current speech LLMs largely perform implicit ASR: on tasks solvable from a transcript, they are behaviorally and mechanistically equivalent to simple Whisper$\to$LLM cascades. We show this through matched-backbone testing across four speech LLMs and six tasks, controlling for the LLM backbone for th…

• Multi-Channel Replay Speech Detection using Acoustic Maps

  Michael Neri, Tuomas Virtanen · 19. Februar 2026

  Replay attacks remain a critical vulnerability for automatic speaker verification systems, particularly in real-time voice assistant applications. In this work, we propose acoustic maps as a novel spatial feature representation for replay speech detection from multi-channel recordings. Derived from …

• Adaptive Rank Allocation for Federated Parameter-Efficient Fine-Tuning of Language Models

  Fei Wu, Jia Hu, Geyong Min, Shiqiang Wang · 19. Februar 2026

  Pre-trained Language Models (PLMs) have demonstrated their superiority and versatility in modern Natural Language Processing (NLP), effectively adapting to various downstream tasks through further fine-tuning. Federated Parameter-Efficient Fine-Tuning (FedPEFT) has emerged as a promising solution to…

• How to Label Resynthesized Audio: The Dual Role of Neural Audio Codecs in Audio Deepfake Detection

  Yixuan Xiao, Florian Lux, Alejandro P\'erez-Gonz\'alez-de-Martos, Ngoc Thang Vu · 19. Februar 2026

  Since Text-to-Speech systems typically don't produce waveforms directly, recent spoof detection studies use resynthesized waveforms from vocoders and neural audio codecs to simulate an attacker. Unlike vocoders, which are specifically designed for speech synthesis, neural audio codecs were originall…

• "Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most

  Kaitlyn Zhou, Martijn Bartelds, Federico Bianchi, James Zou · 18. Februar 2026

  Despite speech recognition systems achieving low word error rates on standard benchmarks, they often fail on short, high-stakes utterances in real-world deployments. Here, we study this failure mode in a high-stakes task: the transcription of U.S. street names as spoken by U.S. participants. We eval…

• "Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most

  Kaitlyn Zhou, Martijn Bartelds, Federico Bianchi, James Zou · 18. Februar 2026

  Despite speech recognition systems achieving low word error rates on standard benchmarks, they often fail on short, high-stakes utterances in real-world deployments. Here, we study this failure mode in a high-stakes task: the transcription of U.S. street names as spoken by U.S. participants. We eval…

• Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction

  Amartyaveer, Murali Kadambi, Chandra Mohan Sharma, Anupam Mondal, Prasanta Kumar Ghosh · 18. Februar 2026

  In this study, we have presented a novel approach to predict the Short-Time Objective Intelligibility (STOI) metric using a bottleneck transformer architecture. Traditional methods for calculating STOI typically requires clean reference speech, which limits their applicability in the real world. To …

• GSRM: Generative Speech Reward Model for Speech RLHF

  Maohao Shen, Tejas Jayashankar, Osama Hanna, Naoyuki Kanda, Yancheng Wang, Kate\v{r}ina \v{Z}mol\'ikov\'a, Ruiming Xie, Niko Moritz, Anfeng Xu, Yashesh Gaur, Gregory Wornell, Qing He, Jilong Wu · 17. Februar 2026

  Recent advances in speech language models, such as GPT-4o Voice Mode and Gemini Live, have demonstrated promising speech generation capabilities. Nevertheless, the aesthetic naturalness of the synthesized audio still lags behind that of human speech. Enhancing generation quality requires a reliable …

• Eureka-Audio: Triggering Audio Intelligence in Compact Language Models

  Dan Zhang, Yishu Lei, Jing Hu, Shuwei He, Songhe Deng, Xianlong Luo, Danxiang Zhu, Shikun Feng, Rui Liu, Jingzhou He, Yu Sun, Hua Wu, Haifeng Wang · 17. Februar 2026

  We present Eureka-Audio, a compact yet high-performance audio language model that achieves competitive performance against models that are 4 to 18 times larger across a broad range of audio understanding benchmarks. Despite containing only 1.7B parameters, Eureka-Audio demonstrates strong performanc…

• SQuTR: A Robustness Benchmark for Spoken Query to Text Retrieval under Acoustic Noise

  Yuejie Li, Ke Yang, Yueying Hua, Berlin Chen, Jianhao Nie, Yueping He, Caixin Kang · 16. Februar 2026

  Spoken query retrieval is an important interaction mode in modern information retrieval. However, existing evaluation datasets are often limited to simple queries under constrained noise conditions, making them inadequate for assessing the robustness of spoken query retrieval systems under complex a…

• Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR

  Jaeyoung Lee, Masato Mimura · 16. Februar 2026

  We present a decoder-only Conformer for automatic speech recognition (ASR) that processes speech and text in a single stack without external speech encoders or pretrained large language models (LLM). The model uses a modality-aware sparse mixture of experts (MoE): disjoint expert pools for speech an…

• "Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most

  Kaitlyn Zhou, Martijn Bartelds, Federico Bianchi, James Zou · 13. Februar 2026

  Despite speech recognition systems achieving low word error rates on standard benchmarks, they often fail on short, high-stakes utterances in real-world deployments. Here, we study this failure mode in a high-stakes task: the transcription of U.S. street names as spoken by U.S. participants. We eval…

• "Sorry, I Didn't Catch That": How Speech Models Miss What Matters Most

  Kaitlyn Zhou, Martijn Bartelds, Federico Bianchi, James Zou · 13. Februar 2026

  Despite speech recognition systems achieving low word error rates on standard benchmarks, they often fail on short, high-stakes utterances in real-world deployments. Here, we study this failure mode in a high-stakes task: the transcription of U.S. street names as spoken by U.S. participants. We eval…

• Moonshine v2: Ergodic Streaming Encoder ASR for Latency-Critical Speech Applications

  Manjunath Kudlur, Evan King, James Wang, Pete Warden · 13. Februar 2026

  Latency-critical speech applications (e.g., live transcription, voice commands, and real-time translation) demand low time-to-first-token (TTFT) and high transcription accuracy, particularly on resource-constrained edge devices. Full-attention Transformer encoders remain a strong accuracy baseline f…

• MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances

  Junhyeok Lee, Helin Wang, Yaohan Guan, Thomas Thebaud, Laureano Moro-Velazquez, Jes\'us Villalba, Najim Dehak · 12. Februar 2026

  We introduce MaskVCT, a zero-shot voice conversion (VC) model that offers multi-factor controllability through multiple classifier-free guidances (CFGs). While previous VC models rely on a fixed conditioning scheme, MaskVCT integrates diverse conditions in a single model. To further enhance robustne…

• VoiceBridge: Designing Latent Bridge Models for General Speech Restoration at Scale

  Chi Zhang, Zehua Chen, Kaiwen Zheng, Jun Zhu · 12. Februar 2026

  Bridge models have recently been explored for speech enhancement tasks such as denoising, dereverberation, and super-resolution, while these efforts are typically confined to a single task or small-scale datasets, with constrained general speech restoration (GSR) capability at scale. In this work, w…

• Self-Supervised Learning for Speaker Recognition: A study and review

  Theo Lepage, Reda Dehak · 12. Februar 2026

  Deep learning models trained in a supervised setting have revolutionized audio and speech processing. However, their performance inherently depends on the quantity of human-annotated data, making them costly to scale and prone to poor generalization under unseen conditions. To address these challeng…