Speech Recognition and Synthesis

Latest papers

• A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition

  Nabil Mosharraf Hossain (Greentech Apps Foundation, United Kingdom), Riasat Islam (Greentech Apps Foundation, United Kingdom, Queen Mary University of London, United Kingdom), Unaizah Obaidellah (University of Malaya, Malaysia) · 19 June 2026

  Quran Automatic Speech Recognition (ASR) aims to convert Quranic recitation into text, enabling applications such as aided memorisation tools and Quranic search engines. However, existing ASR models often exhibit high Word Error Rates (WER) on user-recited verses and lack full coverage of the Qurani…

• Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generation

  Rostislav Makarov, Timo Gerkmann · 19 June 2026

  Classifier guidance is a way to control diffusion generation by using a noise-conditioned classifier to steer the sampling process toward a target class. One drawback of classifier guidance is that it requires two separately trained models: a classifier and a diffusion model. We therefore study a mo…

• FlowEdit: Associative Memory for Lifelong Pronunciation Adaptation in Flow-Matching TTS

  Harshit Singh, Ayush Pratap Singh, Nityanand Mathur · 19 June 2026

  Flow-matching text-to-speech systems achieve remarkable zero-shot quality but remain static after deployment: pronunciation errors on out-of-vocabulary proper nouns persist unless the model is retrained. We introduce FlowEdit, a life-long adaptation framework for frozen flow-matching TTS that learns…

• Systematic Study of Dysarthric Speech Recognition: Spectral Features and Acoustic Models

  Paban Sapkota, Hemant Kumar Kathania, Mikko Kurimo, Sudarsana Reddy Kadiri, Shrikanth Narayanan · 19 June 2026

  The challenge associated with recognizing dysarthric speech primarily arises from pronounced acoustic variability attributed to impaired articulatory precision. Past research has demonstrated improved recognition through the use of hybrid DNN/HMM sequence discriminative training. This paper presents…

• Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech

  Yue Heng Yeo, Haoyang Li, Yizhou Peng, Shreyas Gopal, Hexin Liu, Leibny Paola Garcia-Perera, Hardik B. Sailor, Jeremy H. M. Wong, Eng Siong Chng · 19 June 2026

  Code-switch (CS) Automatic Speech Recognition (ASR) remains challenging due to limited availability of high quality CS text-speech pairs for training. Although synthetic data augmentation via Text-to-speech (TTS) has been explored, existing CS TTS approaches primarily optimise reconstruction fidelit…

• Improving End-to-End Speech Recognition for Dysarthric Speech through In-Domain Data Augmentation

  Paban Sapkota, Hemant Kumar Kathania, Sudarsana Reddy Kadiri, Shrikanth Narayanan · 19 June 2026

  Dysarthric speech recognition is crucial for facilitating effective communication among individuals with dysarthria. However, accurately recognizing dysarthric speech poses significant challenges due to varying severity levels and limited data availability. In this paper, we explore data augmentatio…

• PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors

  Masaya Kawamura, Yuma Shirahata, Kentaro Mitsui, Reo Shimizu · 19 June 2026

  Existing mean opinion score (MOS) prediction models typically predict utterance-level naturalness MOS and can be insensitive to localized pitch-accent errors. We propose Pitch-Accent-focused Speech Quality Assessment (PASQA), which explicitly targets pitch-accent correctness. To train our model, we …

• RIVET: Robust Idempotent Voice Attribute Editing

  Dareen Alharthi, Bhuvan Koduru, Rita Singh, Bhiksha Raj · 19 June 2026

  Voice attribute editing models modify characteristics such as age and gender while preserving speaker identity. In large-scale speech datasets, however, attribute annotations are often noisy or inconsistent, which can cause conditional generative models to produce unstable edits. In this work, we sh…

• FlowFake: Liquid Networks for Audio Deepfake Detection

  Shivaay Dhondiyal, Divyansh Sharma, Dinesh Kumar Vishwakarma · 19 June 2026

  Audio deepfakes generated by neural text-to-speech and voice-cloning systems threaten speaker verification and public discourse at scale. The core challenge is cross-dataset generalization: detectors trained on one synthesis pipeline collapse on unseen forgeries. We argue that this failure is primar…

• ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots

  Ethan Chew, Enjia Wu, Iruss Eng Wei Yeow, Ian Weiqin Lim, Ranen Sim, Brandon Koh Ziheng, Kaleb Nim, Caden Toh Jun Yi, Wei Dong Soin, Darius Kai Keat Koh, Galen King Yu Tay, Prannaya Gupta, Jonathan Ee Fang Koong, Yong Zhi Lim · 18 June 2026

  Air Traffic Control Operators (ATCOs) are vital in ensuring the safe, orderly, and efficient flow of air traffic, yet training capacity is constrained by reliance on specialized human trainers known as simpilots, who must role-play both pilots and ATCOs in a simulated airspace. Existing automated so…

• When Multiple Scripts Matter: Evaluating ASR in Clinical Settings

  Jean Seo, Minkyu Kim, Jeonguk Lee, Jisoo Jung, Wooseok Han, Eunho Yang · 17 June 2026

  Automatic speech recognition (ASR) in non-English clinical settings is challenged by multiscript variability, where the same term may appear in multiple valid orthographic forms. Conventional string-matching evaluation metrics often underestimate ASR performance by treating orthographic variants as …

• L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification

  Hyung-Seok Oh, Deok-Hyeon Cho, Seung-Bin Kim, Seong-Whan Lee · 17 June 2026

  Multilingual speaker verification remains challenging because language-dependent acoustic variability causes speaker identity to become entangled with linguistic characteristics, degrading generalization across languages. In multilingual training, embeddings often encode language cues with speaker i…

• LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization

  Daejin Jo, Jeeyoung Yun, Byungseok Roh, Sungwoong Kim · 16 June 2026

  With the rapid progress of speech language models (SLMs), discrete speech tokens have emerged as a core interface between speech and text, enabling unified modeling across modalities. Recent speech tokenization approaches aim to isolate semantic information from low-level acoustics to better align w…

• Scaling Human and G2P Supervision for Robust Phonetic Transcription

  Alexander Metzger, Aruna Srivastava, Ruslan Mukhamedvaleev · 16 June 2026

  Expert phonetic annotation is costly, especially for non-standard dialects and atypical speech. A common alternative is using Grapheme-to-Phoneme (G2P) models to auto-generate phonetic labels from text transcripts at scale. We study how automatic phonetic transcription performance scales with human …

• HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translation

  Cihan Xiao, Henry Li Xinyuan, Jinyi Yang, Dongji Gao, Matthew Wiesner, Kevin Duh, Sanjeev Khudanpur · 16 June 2026

  We introduce HK-LegiCoST, a new three-way parallel corpus of Cantonese-English translations, containing 600+ hours of Cantonese audio, its standard traditional Chinese transcript, and English translation, segmented and aligned at the sentence level. We describe the notable challenges in corpus prepa…

• An Empirical Study on Learning Latent Representations for Emotional Speech Synthesis

  Vinh Dang Quang, Huy Ngo Quang · 16 June 2026

  For the last couple of years, the field of speech synthesis has improved dramatically thanks to deep learning. There are more and more deep learning-based TTS systems developed to make it possible to produce voices with high intelligibility and naturalness. Meanwhile, controlling the expressiveness …

• Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detection

  Zhuodong Liu, Hugen Lv, Xiangyu Li, Chunhong Yuan · 16 June 2026

  Audio deepfake detectors often fail to generalize across speakers, as they learn speaker-identity features rather than synthesis artifacts, known as implicit identity leakage. Existing methods address this but incur architectural complexity or training instability. This paper proposes a dual-granula…

• Fast When, Careful Who: Dual-Process Multiparty Turn-Taking with Diffusion Augmentation

  Rutherford A. Patamia, Ming Liu, Wei Luo, Favour Ekong, Akan Cosgun · 16 June 2026

  Reliable turn-taking is essential for spoken dialogue systems. However, most existing methods are designed for two-speaker interaction and struggle with realistic multiparty audio containing overlap and rapid speaker changes. We study multiparty turn-taking on the VoxConverse dataset and propose an …

• Robust Spoofed Speech Detection via Temporal Pyramid Modeling

  Mahtab Masoudi Nezhad, Nima Karimian · 16 June 2026

  Spoofed speech detection is increasingly challenged by realistic synthesis, voice conversion, and replay attacks, with cross-dataset generalization remaining a major limitation. This work we propose a Temporal Pyramid Adapter that utilize parallel temporal convolutions with varying receptive fields …

• Probing Low Frame Rate Degradation in Neural Audio Codecs

  Alex Gichamba, Moise Busogi · 16 June 2026

  Low frame rates in neural audio codecs are attractive for autoregressive speech synthesis, where the generation cost scales linearly with the sequence length. Recent work has demonstrated that codecs can operate at 12.5 Hz and below, but the mechanisms underlying low frame rate degradation remain in…

• From Self-Supervised Speech Models to Mixture-of-Experts for Robust Anti-Spoofing

  Hugo Daumain, Driss Matrouf, Khaled Khelif, Mickael Rouvier · 15 June 2026

  Recent advances in speech generation have significantly improved the naturalness of synthetic speech, making spoofing detection increasingly challenging. A key limitation of current anti-spoofing systems is their limited robustness to unseen synthesis methods. In this work, we transform a self-super…

• Mask, Sample, Revise: A Revisable CTMC Inference Stack for Guided Discrete Flow Matching Text-to-Speech

  Alef Iury Siqueira Ferreira, Lucas Rafael Stefanel Gris, Luiz Fernando de Ara\'ujo Vidal, Frederico Santos de Oliveira, Christopher Dane Shulby, Anderson da Silva Soares, Arlindo Rodrigues Galv\~ao Filho · 15 June 2026

  Recent alignment-free non-autoregressive (NAR) text-to-speech (TTS) models formulate synthesis as a conditional infilling task, bypassing explicit duration predictors and external aligners. When speech is represented with neural codec tokens, the infilling problem becomes discrete, making Discrete F…

• MoDiCoL: A Modular Diagnostic Continual Learning Dataset for Robust Speech Recognition

  Theresa Pekarek Rosin, Matthias Kerzel, Stefan Wermter · 15 June 2026

  Modern Automatic Speech Recognition (ASR) systems have made remarkable progress on standard benchmarks, yet performance gaps have emerged under real-world distribution shifts, caused by recording conditions, accents, speech impairments, and noise. Existing datasets and benchmarks typically isolate t…

• Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering

  Haoning Xu, Zhaoqing Li, Huimeng Wang, Youjun Chen, Chengxi Deng, Mengzhe Geng, Xunying Liu · 11 June 2026

  This paper presents a novel data-free and training-free compression approach for speech foundation models using channelwise clustering via k-means. More fine-grained, mixed sparsity pruning by layer-level varying number of parameter clusters is also explored. Experiments conducted on the LibriSpeech…

• Massive Open-Vocabulary Keyword Spotting

  Leonor Barreiros, Raul Monteiro, Afonso Mendes, Gon\c{c}alo M. Correia · 11 June 2026

  Automatic speech recognition systems have been shown to under-perform when it comes to transcribing words rarely seen in the training data, namely specialized terminology. Open-vocabulary keyword spotting, combined with contextual biasing, has been shown to mitigate this issue. However, existing sys…