|
Published Articles >> Table of Contents >> Abstract
Second International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT'04)
pp. 58-65
Speech-Driven Face Synthesis from 3D Video
Ioannis A. Ypsilos, University of Surrey, Guildford, UK
Adrian Hilton, University of Surrey, Guildford, UK
Aseel Turkmani, University of Surrey, Guildford, UK
Philip J. B. Jackson, University of Surrey, Guildford, UK
Full Article Text:
  
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TDPVT.2004.1335143
Send link to a friend
| Abstract |
|
This paper presents a framework for speech-driven synthesis
of real faces from a corpus of 3D video of a person
speaking. Video-rate capture of dynamic 3D face shape and
colour appearance provides the basis for a visual speech
synthesis model. A displacement map representation combines
face shape and colour into a 3D video. This representation
is used to efficiently register and integrate shape
and colour information captured from multiple views. To
allow visual speech synthesis viseme primitives are identified
from the corpus using automatic speech recognition.
A novel non-rigid alignment algorithm is introduced to estimate
dense correspondence between 3D face shape and
appearance for different visemes. The registered displacement
map representation together with a novel optical flow
optimisation using both shape and colour, enables accurate
and efficient non-rigid alignment. Face synthesis from
speech is performed by concatenation of the corresponding
viseme sequence using the non-rigid correspondence
to reproduce both 3D face shape and colour appearance.
Concatenative synthesis reproduces both viseme timing and
co-articulation. Face capture and synthesis has been performed
for a database of 51 people. Results demonstrate
synthesis of 3D visual speech animation with a quality comparable
to the captured video of a person.
|
 Additional Information
|
Citation:
Ioannis A. Ypsilos, Adrian Hilton, Aseel Turkmani, Philip J. B. Jackson,
"Speech-Driven Face Synthesis from 3D Video,"
3dpvt,
pp. 58-65,
Second International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT'04),
2004
|
|
