Construction d'un modèle articulatoire générique multilingue et multilocuteur à l'aide de données IRM temps réel

Keywords

deep learning, computer science, speech processing, applied mathematics, real time MRI, articulatory speech synthesis

Subject details

Context and Rationale Speech production is a complex motor process involving the coordination of articulators (tongue, jaw, lips) to reshape the vocal tract, which in turn defines the acoustic properties of the voice. While modern speech synthesis often focuses on output quality, articulatory synthesis aims to replicate the physical production process itself. This approach is invaluable for understanding phonetic contrasts, simulating speech impairments, and performing acoustic-to-articulatory inversion. Historically, articulatory models have been limited by their reliance on static data or single-speaker/single-language datasets. The MultiSpeech team (Loria) recently advanced this field by using dynamic MRI to model a single speaker's vocal tract. The next frontier, and the focus of this thesis, is to move beyond individual constraints toward a universal model. Research Objectives The primary goal of this PhD project is to construct a generic articulatory model that is independent of both the specific speaker and the language being spoken. The research is divided into two strategic axes: 1. Generic Model Construction: This involves developing an anatomical normalization framework. By neutralizing individual morphological differences, the model can establish a universal control system for articulators that applies to any human anatomy. 2. Multilingual and multispeaker adaptation: The student will map the specific articulatory movements and places of articulation (including complex sounds like diphthongs and affricates) of new languages into the coordinate system of the generic model. Methodology and Innovation The project leverages cutting-edge real-time MRI (rtMRI) data, captured at 50 frames per second. This provides a high-fidelity view of the mid-sagittal plane of the vocal tract during natural speech. Data Acquisition: In collaboration with the IADI laboratory, data from approximately 30 speakers and languages is being collected. Advanced Tracking: To ensure acoustic accuracy, the project will implement state-of-the-art Deep Learning techniques—specifically the nnU-Net framework—to improve the tracking of critical articulators like the tongue tip. Modeling: The student will use deep learning to bridge the gap between phonemic sequences and the geometric evolution of the vocal tract. Scientific Environment The doctoral candidate will operate within a highly complementary environment: IADI (INSERM): World-class expertise in MRI acquisition and anatomical analysis. Loria (MultiSpeech): Leaders in Deep Learning and speech processing. Resources: Access to extensive existing databases (ANR ArtSpeech, Full3FDTalkingHead), high-performance computing clusters, and the ANR ArtAny project funding. Candidate Profile The ideal candidate (master in computer sciences and/or applied mathematics) should possess strong expertise in Deep Learning, applied mathematics, and computer science. Additional knowledge in speech processing or medical imaging (MRI) is highly desirable to navigate this interdisciplinary project. Key Information Supervisor: Yves Laprie Keywords: Articulatory Synthesis, Real-time MRI, Deep Learning, Anatomical Normalization, Speech Processing. Location: Nancy, France (Loria & IADI).