Speech Recognition and Synthesis

Neueste Paper

• FalAR: A Large-scale Speaker-Annotated European Portuguese Speech Corpus of Parliamentary Sessions

  Francisco Teixeira, Carlos Carvalho, Mariana Juli\~ao, Catarina Botelho, Rub\'en Solera-Ure\~na, S\'ergio Paulo, Thomas Rolland, Ben Peters, Isabel Trancoso, Alberto Abad · 27. Mai 2026

  State-of-the-art performance for Automatic Speech Recognition (ASR) largely depends on the availability of large-scale labeled corpora. This creates a demand for increased data collection efforts, particularly for under-represented languages and dialectal varieties. Due to having considerably fewer …

• PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis

  Bowen Li, Shaotong Guo, Zhen Wang, Yang Xiang, Mingli Jin, Yihang Lin, Jiahui Zhao, Weibo Xiong, Dongrui Li, Keming Chen, Yunze Gao, Yuze Zhou, Zeyang Lin, Yue Liu · 27. Mai 2026

  Building state-of-the-art text-to-speech (TTS) systems typically demands millions of hours of proprietary data and complex multi-stage architectures, creating substantial barriers for resource-constrained research teams. In this report, we present PilotTTS, a lightweight autoregressive TTS system th…

• MetaSICL: Adapting Audiroty LLM via Meta Speech In-Context Learning

  Haolong Zheng, Siyin Wang, Zengrui Jin, Mark Hasegawa-Johnson · 27. Mai 2026

  Auditory Large Language Models (LLMs) have demonstrated strong performance across a wide range of speech and audio understanding tasks. Nevertheless, they often struggle when applied to low-resource tasks. In case in-domain labeled data are scarce or mismatched with the true test distribution, direc…

• Raon-Speech Technical Report

  Beomsoo Kim, Changho Choi, Dohyun Kim, Dongki Lee, Ethan Ewer, Eunchong Kim, Gyeongman Kim, Haechan Kim, Hyeonghwan Kim, Inkyu Park, Jihun Yun, Jihwan Moon, Jiyun Kim, Joonghyun Bae, Junhyuck Kim, Minkyu Kim, Sehun Lee, Seungjun Chung, Sungwoo Cho, Dongmin Park, Dongwon Kim, Hara Kang, Jonghyun Lee, Keon Lee, Kangwook Lee, Jaewoong Cho · 26. Mai 2026

  We present Raon-Speech, a top-performing 9B-parameter speech language model (SpeechLM) for English and Korean speech understanding, answering, and generation, and Raon-SpeechChat, a high-performing full-duplex extension for natural real-time conversation. Raon-Speech successfully transforms a pre-tr…

• Continual Speaker Identity Unlearning with Minimal Interference

  Jinju Kim, Yunsung Kang, Gyeong-Moon Park, Jong Hwan Ko · 26. Mai 2026

  Machine unlearning removes designated concepts or knowledge from pre-trained models. Recent work has extended this paradigm to speaker identity unlearning in zero-shot text-to-speech (ZS-TTS), the task of selectively erasing a model's ability to replicate a speaker's voice. Existing methods, however…

• A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning

  Loukas Ilias, Dimitris Askounis · 26. Mai 2026

  Alzheimer's disease (AD) is a progressive neurodegenerative disorder and the leading cause of dementia, affecting memory, reasoning, communication, and daily functioning. Early diagnosis is particularly important, as timely intervention may help slow cognitive decline and improve patient care. Recen…

• WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Pretrained Speech Recognition Transformers

  Akshat Pandey, Karun Kumar, Raphael Tang · 26. Mai 2026

  Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance. In many real-world settings, collecting speech data is impractical, necessitating text-only adaptation. We propose WhisTLE, a deeply supervised, text-only ada…

• Convex Low-resource Accent-Robust Language Detection in Speech Recognition

  Miria Feng, William Tan, Mert Pilanci · 25. Mai 2026

  Globalization and multiculturalism continue to produce increasingly diverse speech varieties. Yet current spoken dialogue systems frequently fail on under-represented dialects and accents, often misidentifying the input language and causing cascading failures in downstream dialogue tasks. Addressing…

• SCRIBE: Diagnostic Evaluation and Rich Transcription Models for Indic ASR

  Kavya Manohar, Arghya Bhattacharya, Kush Juvekar, Kumarmanas Nethil · 22. Mai 2026

  Automatic speech recognition replaces typing only when correction costs less than manual entry, a threshold determined by error types, not counts: fixing a misrecognized domain term costs far more than inserting a comma. Word error rate (WER) fails on two fronts: it collapses distinct error categori…

• Structured Recurrent Mixers for Massively Parallelized Sequence Generation

  Benjamin L. Badger · 20. Mai 2026

  Over the last two decades, language modeling has experienced a shift from the use of predominantly recurrent architectures that process tokens sequentially during training and inference to non-recurrent models that process sequence elements in parallel during training, which results in greater train…

• FormalASR: End-to-End Spoken Chinese to Formal Text

  Wanyi Ning, Yinshang Guo, Haitao Qian, Jiyuan Cheng, Weiyuan Feng, Yufei Zhang · 20. Mai 2026

  Automatic speech recognition (ASR) systems are typically optimized for verbatim transcription, which preserves disfluencies, filler words, and informal spoken structures that are often unsuitable for downstream writing-oriented applications. A common workaround is a two-stage ASR+LLM pipeline for po…

• Benchmarking Commercial ASR Systems on Code-Switching Speech: Arabic, Persian, and German

  Sajjad Abdoli (MAD), Ghassan Al-Sumaidaee (MAD), Clayton W. Taylor (MAD), Ahmad (MAD), ElShiekh, Ahmed Rashad · 20. Mai 2026

  Code-switching -- the natural alternation between two languages within a single utterance -- represents one of the most challenging and under-studied conditions for automatic speech recognition (ASR). Existing commercial ASR benchmarks predominantly evaluate clean, monolingual audio and report a sin…

• Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation

  Zhifei Xie, Kaiyu Pang, Haobin Zhang, Deheng Ye, Xiaobin Hu, Shuicheng Yan, Chunyan Miao · 20. Mai 2026

  Despite rapid advances in automatic speech recognition (ASR) and large audio-language models, robust recognition in real-world environments remains limited by an "acoustic robustness bottleneck": models often lose acoustic grounding and produce omissions or hallucinations under severe, compositional…

• Two-Dimensional Quantization for Geometry-Aware Audio Coding

  Tal Shuster, Eliya Nachmani · 18. Mai 2026

  Recent neural audio codecs have achieved impressive reconstruction quality, typically relying on quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ) and Finite Scalar Quantization (FSQ). However, these quantization techniques limit the geometric structure of the…

• Text-Dependent Speaker Verification (TdSV) Challenge 2024: Team Naive System Report

  Amir Mohammad Rostami, Pourya Jafarzadeh · 15. Mai 2026

  This paper presents a system for the 2024 Text-Dependent Speaker Verification (TdSV) Challenge. The system achieved a Minimum Detection Cost Function (MinDCF) of 0.0461 and an Equal Error Rate (EER) of 1.3\%. Our approach focused on adapting existing state-of-the-art neural networks, ResNet-TDNN and…

• SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning

  KiHyun Nam, Jungwoo Heo, Siu Bae, Ha-Jin Yu, Joon Son Chung · 15. Mai 2026

  As audio-first agents become increasingly common in physical AI, conversational robots, and screenless wearables, audio large language models (audio-LLMs) must integrate speaker-specific understanding to support user authorization, personalization, and context-aware interaction. This requires modeli…

• AudioMosaic: Contrastive Masked Audio Representation Learning

  Hanxun Huang, Qizhou Wang, Xingjun Ma, Cihang Xie, Christopher Leckie, Sarah Erfani · 15. Mai 2026

  Audio self-supervised learning (SSL) aims to learn general-purpose representations from large-scale unlabeled audio data. While recent advances have been driven mainly by generative reconstruction objectives, contrastive approaches remain less explored, partly due to the difficulty of designing effe…

• Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition

  Kush Juvekar, Kavya Manohar, Aditya Srinivas Menon, Arghya Bhattacharya, Kumarmanas Nethil · 14. Mai 2026

  Fine-tuning multilingual ASR models like Whisper for low-resource languages often improves read speech but degrades spontaneous audio performance, a phenomenon we term studio-bias. To diagnose this mismatch, we introduce Vividh-ASR, a complexity-stratified benchmark for Hindi and Malayalam across fo…

• LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training

  Abhishek Moturu, Muhammad Muzammil, Anna Goldenberg, Babak Taati · 14. Mai 2026

  Training deep neural networks with noise and data heterogeneity is a major challenge. We introduce Lightweight Learnable Adaptive Weighting (LiLAW), a method that dynamically adjusts the loss weight of each training sample based on its evolving difficulty, categorized as easy, moderate, and hard, us…

• Does language matter for spoken word classification? A multilingual generative meta-learning approach

  Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der Merwe · 14. Mai 2026

  Meta-learning has been shown to have better performance than supervised learning for few-shot monolingual spoken word classification. However, the meta-learning approach remains under-explored in multilingual spoken word classification. In this paper, we apply the Generative Meta-Continual Learning …

• LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection

  Haohui Zhang, Zhiye Wang, Xiaoying Gan, Xinbing Wang, Bo Jiang · 13. Mai 2026

  Diffusion Language Models (dLLMs) have garnered significant attention for their potential in highly parallel processing. The parallel capabilities of existing dLLMs stem from the assumption of conditional independence at high confidence levels, which ensures negligible discrepancy between the margin…

• Exploring Token-Space Manipulation in Latent Audio Tokenizers

  Francesco Paissan, Luca Della Libera, Mirco Ravanelli, Cem Subakan · 13. Mai 2026

  Neural audio codecs provide compact discrete representations for speech generation and manipulation. However, most codecs organize tokens as frame-level sequences, making it difficult to study or intervene on global factors of variation. In this work, we propose the Latent Audio Tokenizer for Token-…

• Mind the Pause: Disfluency-Aware Objective Tuning for Multilingual Speech Correction with LLMs

  Deepak Kumar, Baban Gain, Asif Ekbal · 13. Mai 2026

  Automatic Speech Recognition (ASR) transcripts often contain disfluencies, such as fillers, repetitions, and false starts, which reduce readability and hinder downstream applications like chatbots and voice assistants. If left unaddressed, such disfluencies can significantly degrade the reliability …

• One Prompt, Many Sounds: Modeling Listener Variability in LLM-Based Equalization

  Ioannis Stylianou, Jon Francombe, Pablo Martinez-Nuevo, Sven Ewan Shepstone, Zheng-Hua Tan · 13. Mai 2026

  Conventional audio equalization is a static process that requires manual and cumbersome adjustments to adapt to changing listening contexts (e.g., mood, location, or social setting). In this paper, we introduce a Large Language Model (LLM)-based alternative that maps natural language text prompts to…

• A Semi-Supervised Framework for Speech Confidence Detection using Whisper

  Adam Wynn, Jingyun Wang · 13. Mai 2026

  Automatic detection of speaker confidence is critical for adaptive computing but remains constrained by limited labelled data and the subjectivity of paralinguistic annotations. This paper proposes a semi-supervised hybrid framework that fuses deep semantic embeddings from the Whisper encoder with a…