Intelligent Robot Distinguished Lecture Series

Vision-Based Modeling and Interaction

Dr. Zhengyou Zhang
Microsoft Research
Redmond, WA

Talk:  Friday, September 28, 2001, 1:50-2:50 p.m.
Room 3105 Engineering Building

Host: J. Weng

Abstract:   We strive to advance the state of the art in 3D computer vision, and develop flexible and robust techniques for ordinary users to gain 3D experience from an ordinary camera. In this talk, I will provide an overview of my research projects.

The first is on 3D object and face modeling from images taken by a free-moving camera. Applications include product advertisement on the Web, virtual conference, and interactive games. We briefly cover the following topics: 

Camera calibration: determine a camera's internal parameters by just observing a planar pattern under unknown orientations. 

Stereo rectification: transform a pair of stereo images based on a geometric meaningful criterion such that the epipolar lines of the rectified images are aligned. This will considerably simplify the stereo matching process. 

Image matching: match points and curves across images within a probabilistic relaxation framework. We can obtain a much richer description of a scene using both points and curves. 

3D photo editing: Virtual or real objects can be inserted in the real images, while preserving the correct 3D information of the scene structure. 

Object modeling: A complete 3D model of an object is built by taking a number of snapshots around the object. 

Face modeling. With five mouse clicks, a 3D face model is built in a few minutes, and the model can be animated immediately. We have successfully built 3D face models for Bill Gates, Steve Ballmer, and many others.  

The second is on vision-based interaction. Applications include human-computer interface and augmented reality. We briefly cover the following topics:  

Visual Screen: A prototype system which converts an ordinary screen into a touch screen. 

Visual Panel: A prototype system which converts a rectangular panel (e.g., an ordinary piece of paper) into a virtual mouse, keyboard and joystick.

Biography:  Zhengyou Zhang is a Senior Researcher with Microsoft Research, Redmond, USA. He received the Ph.D. degree in computer science from the University of Paris XI, France, in 1990, and the Doctor of Science (Habilitation diriger des recherches) diploma from the University of Paris XI, in 1994. He has been with INRIA (French National Institute for Research in Computer Science and Control) for 11 years and was a Senior Research Scientist from 1991 until he joined Microsoft Research in March 1998.  He is an Associate Editor of the "IEEE Transactions on Pattern Analysis and Machine Intelligence" (PAMI) and an Associate Editor of the "International Journal of Pattern Recognition and Artificial Intelligence" (IJPRAI).  He has co-authored the following books: 3D Dynamic Scene Analysis: A Stereo Based Approach (Springer, Berlin, Heidelberg, 1992); Epipolar Geometry in Stereo, Motion and Object Recognition (Kluwer Academic Publishers, 1996); Computer Vision (textbook in Chinese, Chinese Academy of Sciences, 1998).