Michigan State University
Spring 2017
CSE 802 - Pattern Recognition and Analysis, 3 credits

M, W: 12:40 - 2:00 pm, 2245 Engineering Building

Instructor Information

Instructor: Dr. Anil K. Jain
Office: 3143 EB
Office Hours: Tu: 1 pm - 3 pm or by appointment
Phone: 355-9282
Email: jain@cse.msu.edu

Teaching Assistant Information

Inci Baytas (baytasin@msu.edu)
Tarang Chugh (chughtar@msu.edu)

Office: E.B. 3208.
Office hours: M, W: 2 pm - 3 pm or by appointment

Course Information




Introduction

Pattern recognition techniques are used to automatically classify physical objects (handwritten characters, tissue samples, faces) or abstract multidimensional patterns (n points in d dimensions) into known or possibly unknown number of categories. A number of commercial pattern recognition systems are available for character recognition, signature recognition, document classification, fingerprint classification, speech and speaker recognition, white blood cell (leukocyte) classification, military target recognition, etc. Most machine (computer) vision systems employ pattern recognition techniques to identify objects for sorting, inspection, and assembly.

The design of a pattern recognition system consists of following main modules: (i) sensing, (ii) feature extraction, (iii) decision making, and (iv) performance evaluation. The availability of low cost and high resolution sensors (e.g., digital cameras, microphones and scanners) and data sharing over the Internet have resulted in huge repositories of digitized documents (text, speech, image and video). Need for efficient archiving and retrieval of this data has fostered the development of pattern recognition algorithms in new application domains (e.g., text, image and video retrieval, bioinformatics, and face recognition).

A pattern recognition system can be designed based on a number of different approaches: (i) template matching, (ii) geometric (statistical) methods, (iii) structural (syntactic) methods, and (iv) neural (deep) networks. This course will introduce the fundamentals of statistical pattern recognition with examples from several application areas. The course will cover techniques for visualizing and analyzing multi-dimensional data along with algorithms for projection, dimensionality reduction, clustering and classification. The course will present various approaches to classifier design so students can make judicious choices when confronted with real pattern recognition problems. It is important to emphasize that the design of a complete pattern recognition system for a specific application domain (e.g., remote sensing) requires domain knowledge, which is beyond the scope of this course. Students will use available MATLAB tools and will be expected to implement some algorithms using their choice of a programming language.

Prerequisites

CSE 232, MTH 314, and STT 441, or equivalent courses.

An undergraduate level understanding of probability, statistics and linear algebra is assumed. A basic knowledge of MATLAB and Python is essential.


Text Book

Duda, Hart and Stork, Pattern Classification, Second Edition, Wiley, 2001.

Supporting Material

Reading

Tutorials


Course Schedule

Jan 9 Introduction to Pattern Recognition (Ch 1)

Statistical Pattern Recognition: A Review

Lecture - 1 Slides

HW1 assigned, due on Jan 18

Jan 11, 18, 23, 25 Statistical Decision Theory (Ch 2, 2.1 to 2.10)

Lecture - 2, 3, 4, 5 Slides

Jan 18: HW2 assigned, due on Feb 1

Notes on Neyman-Pearson decision rule

Notes on error rate of a linear discriminant function

Jan 30, Feb 1 Parameter Estimation (Ch 3, 3.1 to 3.5, 3.7 to 3.8)

Lecture - 6, 7, 8 Slides

Bayes Estimator for multivariate Gaussian density with unknown covariance matrices

Feb 1: HW3 assigned, due on Feb 15

Feb 6 Component analysis and Discriminants (Ch 3)

Readings on Isomap and LLE

Principle Component Analysis (PCA)

PCA for face Recognition

Feb 8, 13, 15 Neural Networks (Ch 6, 6.1 to 6.3) (Lecture - 9 Slides)

Deep Networks (Lecture - 10, 11 Slides)

Notes on Deep Networks

Discussion of Project

Feb 20 Non-parametric Techniques (Ch 4, 4.1 to 4.3.4)
Feb 22 Non-parametric Techniques (Ch 4, 4.4 to 4.6.1) (Lecture - 12, 13 Slides)
Feb 27 Review of Mid Term Exam
Mar 1 Mid Term Exam

HW4 assigned, due on Mar 22

Mar 6, 8 SPRING BREAK
Mar 13, 15

Curse of Dimensionality (Ch 3.7) (Lecture - 14 Slides)

A Problem of Dimensionality: A Simple Example

Feature Selection (Lecture - 15 Slides)

Feature Selection : Evaluation, Application, and Small Sample Performance by Jain and Zongker

Mar 22, 27

Decision Trees (Ch 8, 8.1 to 8.3) (Lecture - 16 Slides)

Support Vector Machines (Lecture - 17 Slides)

Mar 24: HW5 assigned, due on Apr 10

March 27: Project Progress Report Due

Mar 29, Apr 3 Error Rate Estimation, Bagging, Boosting, Classifier Combination (Ch 9)

Logistic Regression (Lecture - 18 Slides)

Clustering (Lecture - 19-20 Slides)

Apr 5, 10

Unsupervised and semi-supervised learning (Ch. 10)

Apr 10: HW5 due

Apr 12 EXAM-II
Apr 17, 19 Clustering and Multidimensional Scaling (Ch 10) (Lecture - 21-22 Slides)

Data Clustering : 50 Years Beyond K-means (Presentation Slides)

Graph Theoretical Methods for Detecting and Describing Gestalt Clusters by C. Zahn

A Nonlinear Mapping for Data Structure Analysis by J. Sammon

A Global Geometric Framework for Nonlinear Dimensionality Reduction by J. Tanenbaum et al.

Apr 24, 26

April 23: Final Project Presentations Due 11:59 pm

Final Project Presentations

May 1 Final Project Report Due

Grading

Course grade will be assigned based on scores on six homework assignments, two exams and one project. Weights for these three components are as follows:

The cumulative score will be mapped to the letter grade as follows: 90% or higher: 4.0; 85% to 90%: 3.5; 80% to 85%: 3.0 and so on.

Both the exams will be closed book. Makeup exams will be given ONLY if properly justified. Homework solutions must be turned in the class on the date they are due. Late homework solutions will not be accepted.


Academic Integrity

Article 2.3.3 of the Academic Freedom Report states "The student shares with the faculty the responsibility for maintaining the integrity of scholarship, grades, and professional standards." In addition, the Department of Computer Science and Engineering adheres to the policies on academic honesty as specified in General Student Regulations 1.0, Protection of Scholarship and Grades; the all-University Policy on Integrity of Scholarship and Grades; and Ordinance 17.00, Examinations. (See Spartan Life: Student Handbook and Resource Guide)

Therefore, unless authorized by your instructor, you are expected to complete all course assignments, including homework, projects, quizzes, tests and exams, without assistance from any source. You are expected to develop original work for this course; therefore, you may not submit course work you completed for another course to satisfy the requirements for this course. Students who violate MSU academic integrity rules may receive a penalty grade, including a failing grade on the assignment or in the course. Contact your instructor if you are unsure about the appropriateness of your course work. (See also the Academic Integrity webpage.)


Course Project

The purpose of the project is to enable the students to get hands-on experience in the design, implementation and evaluation of pattern recognition algorithms. To facilitate the completion of the project in a semester, it is advised that students work in teams of two.

Download Project Description