Michigan State University
Spring 2017
CSE 802 - Pattern Recognition and Analysis, 3 credits

M, W: 12:40 - 2:00 pm, 2245 Engineering Building

Instructor Information

Instructor: Dr. Anil K. Jain
Office: 3143 EB
Office Hours: Tu: 1 pm - 3 pm or by appointment
Phone: 355-9282
Email: jain@cse.msu.edu

Teaching Assistant Information

Inci Baytas (baytasin@msu.edu)
Tarang Chugh (chughtar@msu.edu)

Office: E.B. 3208.
Office hours: M, W: 2 pm - 3 pm or by appointment

Course Information




Introduction

Pattern recognition techniques are used to automatically classify physical objects (handwritten characters, tissue samples, faces) or abstract multidimensional patterns (n points in d dimensions) into known or possibly unknown number of categories. A number of commercial pattern recognition systems are available for character recognition, signature recognition, document classification, fingerprint classification, speech and speaker recognition, white blood cell (leukocyte) classification, military target recognition, etc. Most machine (computer) vision systems employ pattern recognition techniques to identify objects for sorting, inspection, and assembly.

The design of a pattern recognition system consists of following main modules: (i) sensing, (ii) feature extraction, (iii) decision making, and (iv) performance evaluation. The availability of low cost and high resolution sensors (e.g., digital cameras, microphones and scanners) and data sharing over the Internet have resulted in huge repositories of digitized documents (text, speech, image and video). Need for efficient archiving and retrieval of this data has fostered the development of pattern recognition algorithms in new application domains (e.g., text, image and video retrieval, bioinformatics, and face recognition).

A pattern recognition system can be designed based on a number of different approaches: (i) template matching, (ii) geometric (statistical) methods, (iii) structural (syntactic) methods, and (iv) neural (deep) networks. This course will introduce the fundamentals of statistical pattern recognition with examples from several application areas. The course will cover techniques for visualizing and analyzing multi-dimensional data along with algorithms for projection, dimensionality reduction, clustering and classification. The course will present various approaches to classifier design so students can make judicious choices when confronted with real pattern recognition problems. It is important to emphasize that the design of a complete pattern recognition system for a specific application domain (e.g., remote sensing) requires domain knowledge, which is beyond the scope of this course. Students will use available MATLAB tools and will be expected to implement some algorithms using their choice of a programming language.

Prerequisites

CSE 232, MTH 314, and STT 441, or equivalent courses.

An undergraduate level understanding of probability, statistics and linear algebra is assumed. A basic knowledge of MATLAB and Python is essential.


Text Book

Duda, Hart and Stork, Pattern Classification, Second Edition, Wiley, 2001.

Supporting Material

Reading

Tutorials


Course Schedule

Jan 9 Introduction to Pattern Recognition (Ch 1)

Statistical Pattern Recognition: A Review

Lecture-1 Slides: Pattern Recognition.

HW1 assigned, due on Jan 18

Jan 11, 16, 18, 23 Statistical Decision Theory (Ch 2)

Lecture-2,3 Slides: Bayes Decision Theory

Jan 18: HW2 assigned, due on Jan 30

Notes on Neyman-Pearson decision rule

Notes on error rate of a linear discriminant function

Jan 25, 30, Feb 1 Parameter Estimation (Ch 3)

Bayes Estimator for multivariate Gaussian density with unknown covariance matrices

Bayes Estimator under quadratic loss

Coin Tossing Example

Jan 30: HW3 assigned, due on Feb 13

Feb 6 Curse of Dimensionality (Ch 3)

A Problem of Dimensionality: A Simple Example

Feb 8, 13 Component analysis and Discriminants (Ch 3)

Principle Component Analysis (PCA)

PCA for face Recognition

Feb 13: HW4 assigned, due on March 1

Feb 15, 20, 22 Neural Networks (Ch 6)

Deep Networks

Discussion of Project

Feb 27 Mid Term Exam
Mar 1

Decision Trees (Ch 8)

HW5 assigned, due on Mar 15

Mar 6, 8 SPRING BREAK
Mar 13, 15

Non-parametric Techniques (Ch 4)

A Branch and Bound Algorithm for Computing k-Nearest Neighbors

Mar 15: HW6 assigned due on Mar 29

Mar 22, 27 Linear Discriminant functions (Ch 5)

Support Vector Machines

Mar 29, Apr 3 Error Rate Estimation, Bagging, Boosting, Classifier Combination (Ch 9)

Mar 29: HW6 due

Apr 5 Feature Selection

Feature Selection : Evaluation, Application, and Small Sample Performance by Jain and Zongker

April 5: Project Progress Report Due

Apr 12, 17, 19 Unsupervised Learning, Clustering, semi-supervised learning and Multidimensional Scaling (Ch 10)

Data Clustering : 50 Years Beyond K-means (Presentation Slides)

Graph Theoretical Methods for Detecting and Describing Gestalt Clusters by C. Zahn

A Nonlinear Mapping for Data Structure Analysis by J. Sammon

A Global Geometric Framework for Nonlinear Dimensionality Reduction by J. Tanenbaum et al.

Apr 24, 26

April 23: Final Project Presentations Due 11:59 pm

Final Project Presentations

April 26: Final Project Report Due

May 4 FINAL EXAM, 12:45pm - 2:45pm, 2245 EB

Grading

Course grade will be assigned based on scores on six homework assignments, two exams and one project. Weights for these three components are as follows:

The cumulative score will be mapped to the letter grade as follows: 90% or higher: 4.0; 85% to 90%: 3.5; 80% to 85%: 3.0 and so on.

Both the exams will be closed book. Makeup exams will be given ONLY if properly justified. Homework solutions must be turned in the class on the date they are due. Late homework solutions will not be accepted.


Academic Integrity

Article 2.3.3 of the Academic Freedom Report states "The student shares with the faculty the responsibility for maintaining the integrity of scholarship, grades, and professional standards." In addition, the Department of Computer Science and Engineering adheres to the policies on academic honesty as specified in General Student Regulations 1.0, Protection of Scholarship and Grades; the all-University Policy on Integrity of Scholarship and Grades; and Ordinance 17.00, Examinations. (See Spartan Life: Student Handbook and Resource Guide)

Therefore, unless authorized by your instructor, you are expected to complete all course assignments, including homework, projects, quizzes, tests and exams, without assistance from any source. You are expected to develop original work for this course; therefore, you may not submit course work you completed for another course to satisfy the requirements for this course. Students who violate MSU academic integrity rules may receive a penalty grade, including a failing grade on the assignment or in the course. Contact your instructor if you are unsure about the appropriateness of your course work. (See also the Academic Integrity webpage.)


Course Project

The purpose of the project is to enable the students to get hands-on experience in the design, implementation and evaluation of pattern recognition algorithms. To facilitate the completion of the project in a semester, it is advised that students work in teams of two.

Download Project Description