Mobile Face Capture System

Abstract

Capturing a face in a multi-user collaborative environment has been a problem of concern for several years. The problem becomes more severe in mobile applications where the capturing system may occlude the user's field of view. In this thesis, a system is proposed that captures two side views of the face simultaneously and generates a frontal view. Applications of facial capture system include tele-conferencing, wearable computing, collaborative work and other mobile applications. The system is designed to produce in real-time a stable, quality frontal view of an HMD user whose face is captured with little obstruction of the field of view. The frontal views are generated by warping and blending of the side views after a calibration step. In tests, the generated views compared well with real video based on both normalized cross correlation and the Euclidean distance between some of the prominent feature points. Preliminary qualitative assessment of these generated views also concludes that the generated videos adequate to support the intended applications. A 3D face model that can support the generation of arbitrary views was also constructed.
 

Keywords  Face capture, Calibration, View synthesis, Tele-presence, Collaborative work,  Structured light, Video processing, Face warping, Face mosaicking.

Results (10 Meg files to download)
[ Left Face Video ]  [ Right Face Video ]  [ Virtual Frontal Face Video ] [ Real Frontal Face Video ]                                  

 

Faculty

Dr. George Stockman, Department of Computer Science and engineering, Michigan State University. (Computer Vision, Facial Animation)

Dr. Frank Biocca, Department of Telecommunications, Michigan State University. (Human-Computer Interaction, Tele-Presence)

Dr. Jannick Rolland, School of Optics / CREOL and School of EE & CS, University of Central Florida. (Optics, Visualization)

Students

Chandan Reddy

Miguel Figueroa

 

Publications / Reports

1. Chandan K. Reddy, George C. Stockman, Jannick P. Rolland and Frank A. Biocca,  “Mobile Face Capture for Virtual Face Videos”, Proceedings of first IEEE workshop on Face Processing in Video, July 2004.

2.  Chandan Reddy, “A Non-obtrusive Head Mounted Face Capture System”, Master’s Thesis, Dept. of Computer Science & Engineering, Michigan State University, August 2003. [ PPT Slides ]

Presentations / Posters

1. Chandan Reddy, “A Non-obtrusive Head Mounted Face Capture System”, School of Optics / CREOL, University of Central Florida, July 2003.

2. Chandan Reddy, “A Face-to-Face Communication System using Augmented Reality”, PRIP Lab, Dept. of Computer Science & Engineering, Michigan State University, October 2002. [ Abstract  ]

 

Last Updated: October 10, 2003                                                                                                       Copyright © 2003 Chandan Reddy, All Rights Reserved