Speech Recognition and Synthesis

Derniers papiers

• Sparsity-Inducing Divergence Losses for Biometric Verification

  Dimitrios Koutsianos, Ladislav Mo\v{s}ner, Yannis Panagakis, Themos Stafylakis · 1 juillet 2026

  Performance in face and speaker verification is largely driven by margin-penalty softmax losses such as CosFace and ArcFace. Recently introduced $\alpha$-divergence loss functions offer a compelling alternative, particularly due to their ability to induce sparse solutions (when $\alpha>1$). However,…

• Linguistic Bias Mitigation for Spoofing Detection via Gradient Reversal and A Variational Information Bottleneck

  Anh-Tuan Dao, Driss Matrouf, Mickael Rouvier, Nicholas Evans · 1 juillet 2026

  Rapid advancements in generative speech technology have compromised the reliability of voice biometrics. While current spoofing detectors excel when assessed under in-domain conditions, generalisation to out-of-domain settings is often poor. We show that this can be due to linguistic bias. A relianc…

• UniSAE: Unified Speech Attribute Editing on Speaker, Emotion and Low-Level Content via Discrete Phonetic Posteriorgram Modelling

  Chuanbo Zhu, Wuyou Zhou, Rongxiu Zhong, Shilei Zhang, Kun Qian, Yike Guo, Wei Xue · 1 juillet 2026

  Speech editing aims to modify specific portions of an utterance while preserving the remaining speech. Existing approaches primarily focus on word-level content modification and typically treat content, speaker, and emotion editing as separate tasks, limiting both editing granularity and flexibility…

• Listening Between the Lines: Joint Learning of ASR Embeddings and LLM-Augmented Linguistics for Dementia Detection

  Olivier Jiyoun Jung, Jonghyeon Park, Myungwoo Oh · 1 juillet 2026

  Early detection of dementia through speech analysis offers a non-invasive screening alternative, but capturing both acoustic and linguistic biomarkers remains challenging. We propose a multimodal framework leveraging Whisper for dual-purpose extraction: acoustic representations from encoder outputs …

• LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

  Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo Oh · 30 juin 2026

  Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous speech offers a non-invasive screening modality. Conventional approaches often focus on a single representational dimension -- such as acoustic descriptors, pause modeling, automatic speech reco…

• OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL

  Karl El Hajal, Mathew Magimai. -Doss · 30 juin 2026

  We propose Online Latent prediction with Invariant Views and rEconstruction (OLIVE), a self-supervised speech representation learning framework that jointly optimizes analysis and synthesis objectives. OLIVE combines view-augmented masked latent prediction with waveform reconstruction under a unifie…

• Geometrically Principled Randomized Optimization for Efficient LLM Training

  Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla · 30 juin 2026

  Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches employed primarily for computational efficiency. In this work, we question the intuition behind why random projections a…

• AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification

  Chuxiao Zuo, Yao Zhu, Minqiang Xu, Manhong Wang, Yunke Zhang, Fei Huang · 30 juin 2026

  Multimodal speaker identification systems face two key challenges in real-world deployment: missing modalities and language mismatch between training and testing conditions. In practical scenarios, background multi-speaker conversations, ambient noise, and overlapping speech further degrade identifi…

• Advancing Speaker-Based Vocal Effort Classification with WavLM and Data Augmentation in Naturalistic Non-Calibrated Speech Recordings

  Zahra Omidi, John H. L. Hansen · 29 juin 2026

  The variations in vocal effort range (e.g. whisper, soft, neutral, loud, shout) alter production and speech acoustics, reducing intelligibility and limiting the robustness of any subsequent speech technology. Classification is challenging since effort lies on a continuum, adjacent categories are eas…

• HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models

  Artem Ploujnikov, Francesco Verdini, Samir Sadok, Mirco Ravanelli · 29 juin 2026

  Discrete audio representations have become increasingly popular for building multimodal text-audio systems and integrating audio capabilities into Large Language Models (LLMs). However, numerous studies report performance degradation on various downstream tasks due to information loss during discret…

• What Was That Again? Certified Robustness for Automatic Speech Recognition

  Andrew C. Cullen, Neil Marchant, Jiani Xie, Paul Montague, Benjamin I. P. Rubinstein · 29 juin 2026

  Automatic Speech Recognition systems are notoriously both sensitive to adversarial and benign perturbations. While this has been repeatedly demonstrated using reference datasets, detecting such behaviors in deployed systems is incredibly challenging, due to the absence of oracle knowledge of the tru…

• DG^VoiC: Speaker Clustering for Fraud Investigation under Real Call-Centre Conditions

  Muhammad Shakeel Akram, Amal Htait, Abdul Hamid Sadka, Emma Meisingseth, Karishma Jaitly · 29 juin 2026

  Insurance fraud remains costly and operationally difficult, particularly in call-centre workflows where many customer interactions begin at FNOL. While recent fraud detection methods mainly rely on structured data, text, or images, repeated speaker identity across calls remains underused as an inves…

• VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation

  Tianxin Xie, Chenxing Li, Dong Yu, Li Liu · 26 juin 2026

  Recently, zero-shot text-to-speech (TTS) has enabled high-fidelity and expressive speech synthesis, but it often fails to imitate unseen speaking styles from uncommon scenarios (e.g., crosstalk, dialects). Moreover, fine-tuning pretrained models requires large, high-quality datasets, limiting rapid …

• Tuning Language Models by Mixture-of-Depths Ensemble

  Haoyan Luo, Lucia Specia · 26 juin 2026

  Transformer-based Large Language Models (LLMs) traditionally rely on final-layer loss for finetuning and final-layer representations for predictions, potentially overlooking the predictive power embedded in late layers. Interpretability tools such as the logit lens show that late-layer representatio…

• Neural Speaker Diarization via Multilingual Training: Evaluation on Low-Resource Nepali-Hindi Speech

  Samip Neupane, Sandesh Pokhrel, Sandesh Pyakurel, Basanta Joshi · 26 juin 2026

  Speaker diarization, the task of determining "who spoke when" in a multi-speaker recording, is a critical component in applications such as meeting transcription, accessibility tools, and multilingual information retrieval. While end-to-end neural diarization systems have achieved strong performance…

• STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity

  Sitong Cheng, Weizhen Bian, Songjun Cao, Jin Li, Bei Liu, Chunyang Jiang, Yike Zhang, Weihao Wu, Yiming Li, Chi-Min Chan, Long Ma, Wei Xue · 25 juin 2026

  Speech-to-speech translation (S2ST) should preserve not only lexical meaning, but also expressive attributes: emotion, scenario style (e.g., news reporting vs. dramatic dialogue), and nonverbal vocalizations (NVs). Moreover, collecting cross-lingual target speech that is both translation-faithful an…

• CrossAccent-TTS: Cross-Lingual Accent-Intensity Controllable Text-to-Speech via Disentangled Speaker and Accent Representations

  Ram Annamdevula, Ankit Tatawat, Ashishkumar P. Gudmalwar, Nirmesh J. Shah, Pankaj Wasnik · 25 juin 2026

  Accent conversion and controllability remain fundamental challenges in cross-lingual text-to-speech (TTS), particularly for low-resource and phonetically diverse Indic languages. While recent large language model (LLM)-based TTS systems exhibit strong cross-lingual generalization, they provide limit…

• Error-Aware TF-IDF Retrieval-Augmented Generation for ASR Error Correction

  Mohammad Aref Jafari-Raddani · 25 juin 2026

  End-to-end automatic speech recognition systems frequently hallucinate rare entities and domain-specific terms, especially in low-resource languages. While retrieval-augmented generation frameworks can mitigate these errors using large language models, current architectures face significant challeng…

• Phoneme-Level Mispronunciation Screening in Polish-Speaking Children with an Explainable Assistant

  Milosz Dudek, Daria Hemmerling, Kamil Kwarciak, Maciej Stroinski, Maria Pensko, Mateusz Kowalewski, Leonid Pavlovskyi, Sebastian Jurczak, Anna-Mariia Vitkovska, Zuzanna Miodonska, Natalia Mocko, Michal Krecichwost · 25 juin 2026

  Early identification of speech sound errors in children is often limited by access to specialists, motivating lightweight screening tools that can operate outside the clinic. We present a screening pipeline for Polish-speaking children focused on sibilant substitutions, coupling a wav2vec2-based CTC…

• Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR

  Nenad Banfic · 24 juin 2026

  Adapting a streaming speech recognition model to a new language requires choosing between two plausible warm starts: a multilingual (ML) encoder or an English-only (EN) encoder. The common intuition is that the multilingual encoder should help most at low data, but it is unclear how long that advant…

• ZONOS2 Technical Report

  Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close, Beren Millidge · 24 juin 2026

  We present ZONOS2 8B, our latest TTS model, which achieves state-of-the-art naturalness, prosody, and voice cloning fidelity. We improve upon Zonos-v0.1 across scale, data, and training recipe. We scale the model from 1.6B to 8B total parameters (900M active) with a novel mixture-of-experts (MoE) ba…

• How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures

  Abhijit Sinha, Hemant Kumar Kathania, Mohit Joshi, Harishankar Kumar, Shrikanth Narayanan, Sudarsana Reddy Kadiri · 23 juin 2026

  Self-supervised learning (SSL) models have become a central component of modern speech processing systems, as they enable the learning of rich acoustic representations without reliance on labeled data. Despite their success on adult speech, it remains unclear how effectively these models capture spe…

• Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verification

  Mickael Rouvier, Pierre Michel Bousquet · 23 juin 2026

  In this paper, we present Kiwano, an open-source toolkit designed to advance research and evaluation for speaker verification. Kiwano provides a lightweight yet extensible framework built on PyTorch, offering standardized recipes, pretrained models, and integration of several widely used speaker ver…

• Interleaved Speech Language Models Latently Work In Text

  Talia Sternberg, Gallil Maimon, Yossi Adi · 23 juin 2026

  Speech language models (SLMs) have been extensively studied, with the common paradigm incorporating text data and pre-trained text LMs. A leading approach is speech-text interleaving in which models are trained over sequences containing both speech and text tokens, aiming to boost even speech-only c…

• The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recovery

  Ivan Novosad · 23 juin 2026

  We study the limits of CTC-internal scoring for N-best hypothesis selection and locate the information bottleneck separating acoustic confidence from linguistic plausibility. Eleven CTC-internal and acoustic-feature scoring strategies produce no statistically significant WER improvement over greedy …