Speech Recognition and Synthesis

Neueste Paper

• MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates

  Zikang Huang, Meng Ge, Tianrui Wang, Xuanchen Li, Xiaobao Wang, Longbiao Wang, Jianwu Dang · 25. März 2026

  Self-supervised learning (SSL) has advanced speech processing. However, existing speech SSL methods typically assume a single sampling rate and struggle with mixed-rate data due to temporal resolution mismatch. To address this limitation, we propose MSRHuBERT, a multi-sampling-rate adaptive pre-trai…

• Selective Classifier-free Guidance for Zero-shot Text-to-speech

  John Zheng, Farhad Maleki · 25. März 2026

  In zero-shot text-to-speech, achieving a balance between fidelity to the target speaker and adherence to text content remains a challenge. While classifier-free guidance (CFG) strategies have shown promising results in image generation, their application to speech synthesis are underexplored. Separa…

• Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic AudioLLMs

  Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury · 24. März 2026

  Audio large language models (LLMs) enable unified speech understanding and generation, but adapting them to linguistically complex and dialect-rich settings such as Arabic-English remains challenging. We present a controlled study of multi-task instruction tuning for an Arabic-centric audio LLM acro…

• Voice Privacy from an Attribute-based Perspective

  Mehtab Ur Rahman, Martha Larson, Cristian Tejedor Garc\'ia · 24. März 2026

  Voice privacy approaches that preserve the anonymity of speakers modify speech in an attempt to break the link with the true identity of the speaker. Current benchmarks measure speaker protection based on signal-to-signal comparisons. In this paper, we introduce an attribute-based perspective, where…

• Abjad-Kids: An Arabic Speech Classification Dataset for Primary Education

  Abdul Aziz Snoubara, Baraa Al_Maradni, Haya Al_Naal, Malek Al_Madrmani, Roaa Jdini, Seedra Zarzour, Khloud Al Jallad · 24. März 2026

  Speech-based AI educational applications have gained significant interest in recent years, particularly for children. However, children speech research remains limited due to the lack of publicly available datasets, especially for low-resource languages such as Arabic.This paper presents Abjad-Kids,…

• SNAP: Speaker Nulling for Artifact Projection in Speech Deepfake Detection

  Kyudan Jung, Jihwan Kim, Minwoo Lee, Soyoon Kim, Jeonghoon Kim, Jaegul Choo, Cheonbok Park · 24. März 2026

  Recent advancements in text-to-speech technologies enable generating high-fidelity synthetic speech nearly indistinguishable from real human voices. While recent studies show the efficacy of self-supervised learning-based speech encoders for deepfake detection, these models struggle to generalize ac…

• HELIX: Scaling Raw Audio Understanding with Hybrid Mamba-Attention Beyond the Quadratic Limit

  Khushiyant, Param Thakkar · 24. März 2026

  Audio representation learning typically evaluates design choices such as input frontend, sequence backbone, and sequence length in isolation. We show that these axes are coupled, and conclusions from one setting often do not transfer to others. We introduce HELIX, a controlled framework comparing pu…

• LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource Automatic Speech Recognition Across Language Families

  Jianan Chen, Xiaoxue Gao, Tatsuya Kawahara, Nancy F. Chen · 23. März 2026

  Large language models (LLMs) have driven substantial advances in speech language models (SpeechLMs), yielding strong performance in automatic speech recognition (ASR) under high-resource conditions. However, existing benchmarks predominantly focus on high-resource languages, leaving the ASR behavior…

• Demonstration of Adapt4Me: An Uncertainty-Aware Authoring Environment for Personalizing Automatic Speech Recognition to Non-normative Speech

  Niclas Pokel, Yiming Zhao, Pehu\'en Moure, Yingqiang Gao, Roman B\"ohringer · 23. März 2026

  Personalizing Automatic Speech Recognition (ASR) for non-normative speech remains challenging because data collection is labor-intensive and model training is technically complex. To address these limitations, we propose Adapt4Me, a web-based decentralized environment that operationalizes Bayesian a…

• Breeze Taigi: Benchmarks and Models for Taiwanese Hokkien Speech Recognition and Synthesis

  Yu-Siang Lan, Chia-Sheng Liu, Yi-Chang Chen, Po-Chun Hsu, Allyson Chiu, Shun-Wen Lin, Da-shan Shiu, Yuan-Fu Liao · 23. März 2026

  Taiwanese Hokkien (Taigi) presents unique opportunities for advancing speech technology methodologies that can generalize to diverse linguistic contexts. We introduce Breeze Taigi, a comprehensive framework centered on standardized benchmarks for evaluating Taigi speech recognition and synthesis sys…

• ProKWS: Personalized Keyword Spotting via Collaborative Learning of Phonemes and Prosody

  Jianan Pan, Yuanming Zhang, Kejie Huang · 20. März 2026

  Current keyword spotting systems primarily use phoneme-level matching to distinguish confusable words but ignore user-specific pronunciation traits like prosody (intonation, stress, rhythm). This paper presents ProKWS, a novel framework integrating fine-grained phoneme learning with personalized pro…

• Enhancing Multi-Corpus Training in SSL-Based Anti-Spoofing Models: Domain-Invariant Feature Extraction

  Anh-Tuan Dao, Driss Matrouf, Mickael Rouvier, Nicholas Evans · 20. März 2026

  The performance of speech spoofing detection often varies across different training and evaluation corpora. Leveraging multiple corpora typically enhances robustness and performance in fields like speaker recognition and speech recognition. However, our spoofing detection experiments show that multi…

• MOSS-TTS Technical Report

  Yitian Gong, Botian Jiang, Yiwei Zhao, Yucheng Yuan, Kuangwei Chen, Yaozhou Jiang, Cheng Chang, Dong Hong, Mingshu Chen, Ruixiao Li, Yiyang Zhang, Yang Gao, Hanfu Chen, Ke Chen, Songlin Wang, Xiaogui Yang, Yuqian Zhang, Kexin Huang, ZhengYuan Lin, Kang Yu, Ziqi Chen, Jin Wang, Zhaoye Fei, Qinyuan Cheng, Shimin Li, Xipeng Qiu · 20. März 2026

  This technical report presents MOSS-TTS, a speech generation foundation model built on a scalable recipe: discrete audio tokens, autoregressive modeling, and large-scale pretraining. Built on MOSS-Audio-Tokenizer, a causal Transformer tokenizer that compresses 24 kHz audio to 12.5 fps with variable-…

• PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting

  Jianan Pan, Kejie Huang · 20. März 2026

  As advancements in technologies like Internet of Things (IoT), Automatic Speech Recognition (ASR), Speaker Verification (SV), and Text-to-Speech (TTS) lead to increased usage of intelligent voice assistants, the demand for privacy and personalization has escalated. In this paper, we introduce a mult…

• Impact of automatic speech recognition quality on Alzheimer's disease detection from spontaneous speech: a reproducible benchmark study with lexical modeling and statistical validation

  Himadri Samanta · 20. März 2026

  Early detection of Alzheimer's disease from spontaneous speech has emerged as a promising non-invasive screening approach. However, the influence of automatic speech recognition (ASR) quality on downstream clinical language modeling remains insufficiently understood. In this study, we investigate Al…

• Modeling Overlapped Speech with Shuffles

  Matthew Wiesner, Samuele Cornell, Alexander Polok, Lucas Ondel Yang, Luk\'a\v{s} Burget, Sanjeev Khudanpur · 19. März 2026

  We propose to model parallel streams of data, such as overlapped speech, using shuffles. Specifically, this paper shows how the shuffle product and partial order finite-state automata (FSAs) can be used for alignment and speaker-attributed transcription of overlapped speech. We train using the total…

• Attention-guided Evidence Grounding for Spoken Question Answering

  Ke Yang, Bolin Chen, Yuejie Li, Yueying Hua, Jianhao Nie, Yueping He, Bowen Li, Chengjun Mao · 18. März 2026

  Spoken Question Answering (Spoken QA) presents a challenging cross-modal problem: effectively aligning acoustic queries with textual knowledge while avoiding the latency and error propagation inherent in cascaded ASR-based systems. In this paper, we introduce Attention-guided Evidence Grounding (AEG…

• SyncSpeech: Efficient and Low-Latency Text-to-Speech based on Temporal Masked Transformer

  Zhengyan Sheng, Zhihao Du, Shiliang Zhang, Zhijie Yan, Liping Chen · 17. März 2026

  Current text-to-speech (TTS) models face a persistent limitation: autoregressive (AR) models suffer from low generation efficiency, while modern non-autoregressive (NAR) models experience high latency due to their unordered temporal nature. To bridge this divide, we introduce SyncSpeech, an efficien…

• Boosting Large Language Models with Mask Fine-Tuning

  Mingyuan Zhang, Yue Bai, Huan Wang, Yizhou Wang, Qihua Dong, Yitian Zhang, Yun Fu · 17. März 2026

  The large language model (LLM) is typically integrated into the mainstream optimization protocol. No work has questioned whether maintaining the model integrity is \textit{indispensable} for promising performance. In this work, we introduce Mask Fine-Tuning (MFT), a novel LLM fine-tuning paradigm de…

• SloPal: A 60-Million-Word Slovak Parliamentary Corpus with Aligned Speech and Fine-Tuned ASR Models

  Erik Bo\v{z}\'ik, Marek \v{S}uppa · 17. März 2026

  Slovak remains a low-resource language for automatic speech recognition (ASR), with fewer than 100 hours of publicly available training data. We present SloPal, a comprehensive Slovak parliamentary corpus comprising 330,000 speaker-segmented transcripts (66 million words, 220 million tokens) spannin…

• Data-Efficient ASR Personalization for Non-Normative Speech Using an Uncertainty-Based Phoneme Difficulty Score for Guided Sampling

  Niclas Pokel, Pehu\'en Moure, Roman B\"ohringer, Yingqiang Gao · 17. März 2026

  ASR systems struggle with non-normative speech due to high acoustic variability and data scarcity. We propose a data-efficient method using phoneme-level uncertainty to guide fine-tuning for personalization. Instead of computationally expensive ensembles, we leverage Variational Low-Rank Adaptation …

• A Hierarchical End-of-Turn Model with Primary Speaker Segmentation for Real-Time Conversational AI

  Karim Helwani, Hoang Do, James Luan, Sriram Srinivasan · 17. März 2026

  We present a real-time front-end for voice-based conversational AI to enable natural turn-taking in two-speaker scenarios by combining primary speaker segmentation with hierarchical End-of-Turn (EOT) detection. To operate robustly in multi-speaker environments, the system continuously identifies and…

• Localizing and Editing Knowledge in Large Audio-Language Models

  Sung Kyun Chung, Jiaheng Dong, Qiuchi Hu, Gongping Huang, Hong Jia, Ting Dang · 17. März 2026

  Large Audio-Language Models (LALMs) have shown strong performance in speech understanding, making speech a natural interface for accessing factual information. Yet they are trained on static corpora and may encode incorrect facts. Existing model editing methods localize and update facts in text-only…

• VoXtream2: Full-stream TTS with dynamic speaking rate control

  Nikita Torgashov, Gustav Eje Henter, Gabriel Skantze · 17. März 2026

  Full-stream text-to-speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtrea…

• Investigating the Impact of Speech Enhancement on Audio Deepfake Detection in Noisy Environments

  Anacin, Angela, Shruti Kshirsagar, Anderson R. Avila · 17. März 2026

  Logical Access (LA) attacks, also known as audio deepfake attacks, use Text-to-Speech (TTS) or Voice Conversion (VC) methods to generate spoofed speech data. This can represent a serious threat to Automatic Speaker Verification (ASV) systems, as intruders can use such attacks to bypass voice biometr…