Distinguished Lecture Series

Vision-Based Modeling and Interaction

Dr. Zhenyou Zhang
Microsoft Corporation
Redmond, Washington

Talk: Friday, September 28, 11am-noon
Room 3105 Engineering Building

Abstract: We strive to advance the state of the art in 3D computer vision, and develop flexible and robust techniques for ordinary users to gain 3D experience from an ordinary camera. In this talk, I will provide an overview of my research projects.

The first is on 3D object and face modeling from images taken by a free-moving camera. Applications include product advertisement on the Web, virtual conference, and interactive games. We briefly cover the following topics:

Camera calibration: determine a camera's internal parameters by just observing a planar pattern under unknown orientations.
Stereo rectification: transform a pair of stereo images based on a geometric meaningful criterion such that the epipolar lines of the rectified images are aligned. This will considerably simplify the stereo matching process.
Image matching: match points and curves across images within a probabilistic relaxation framework. We can obtain a much richer description of a scene using both points and curves.
3D photo editing: Virtual or real objects can be inserted in the real images, while preserving the correct 3D information of the scene structure.
Object modeling: A complete 3D model of an object is built by taking a number of snapshots around the object.
Face modeling. With five mouse clicks, a 3D face model is built in a few minutes, and the model can be animated immediately. We have successfully built 3D face models for Bill Gates, Steve Ballmer, and many others.

The second is on vision-based interaction. Applications include human-computer interface and augmented reality. We briefly cover the following topics:

Visual Screen: A prototype system which converts an ordinary screen into a touch screen.
Visual Panel: A prototype system which converts a rectangular panel (e.g., an ordinary piece of paper) into a virtual mouse, keyboard and joystick.

Biography: Zhengyou Zhang is a Senior Researcher with Microsoft Research, Redmond, USA. He received the B.S. degree in electronic engineering from the University of Zhejiang, China, in 1985, the M.S. in computer science from the University of Nancy, France, in 1987, the Ph.D. degree in computer science from the University of Paris XI, France, in 1990, and the Doctor of Science (Habilitation à diriger des recherches) diploma from the University of Paris XI, in 1994. He has been with INRIA (French National Institute for Research in Computer Science and Control) for 11 years and was a Senior Research Scientist from 1991 until he joined Microsoft Research in March 1998. In 1996-1997, he spent one-year sabbatical as an Invited Researcher at the Advanced Telecommunications Research Institute International (ATR), Kyoto, Japan. His current research interests include 3D computer vision, dynamic scene analysis, vision and graphics, facial image analysis, and visual learning. He is an Associate Editor of the "IEEE Transactions on Pattern Analysis and Machine Intelligence" (PAMI) and an Associate Editor of the "International Journal of Pattern Recognition and Artificial Intelligence" (IJPRAI). He is a Senior Member of the IEEE. He holds guest or adjunct faculty positions at Zhejiang University, Institute of Automation (Chinese Academy of Sciences), Northern Jiaotong University, and University of Southern California. He has co-authored the following books: 3D Dynamic Scene Analysis: A Stereo Based Approach (Springer, Berlin, Heidelberg, 1992); Epipolar Geometry in Stereo, Motion and Object Recognition (Kluwer Academic Publishers, 1996); Computer Vision (textbook in Chinese, Chinese Academy of Sciences, 1998).