Replacing speaker-independent recognition task with speaker-dependent task for lip-reading using First Order Motion Model

Michinari Kodama; Takeshi Saitoh

doi:10.1117/12.2623640

16 February 2022 Replacing speaker-independent recognition task with speaker-dependent task for lip-reading using First Order Motion Model

Michinari Kodama, Takeshi Saitoh

Author Affiliations +

Proceedings Volume 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021); 1208323 (2022) https://doi.org/10.1117/12.2623640
Event: Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 2021, Kunming, China

Abstract

There is a tendency to deal with a speaker-independent recognition task in the lip-reading field by collecting speech scenes from many speakers. The data collection task is time-consuming. This paper proposes a method to solve this problem. According to a driving video, First Order Motion Model (FOMM) is a deep generative model that generates a video sequence from a source image. Our idea is to apply FOMM to all speech scenes in the dataset to generate the speech scenes recording from one speaker. We propose a preprocessing method to replace the speaker-independent recognition task with the speaker-dependent recognition task by applying FOMM. We applied the proposed method to two publicly available databases: OuluVS and CUAVE, and confirmed that the recognition accuracy was improved by applying the proposed method to both databases.

Citation Download Citation

Michinari Kodama and Takeshi Saitoh "Replacing speaker-independent recognition task with speaker-dependent task for lip-reading using First Order Motion Model", Proc. SPIE 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 1208323 (16 February 2022); https://doi.org/10.1117/12.2623640

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available