Pattern recognition techniques are used to automatically classify physical objects (handwritten characters, tissue samples, faces) or abstract multidimensional patterns (n points in d dimensions) into known or possibly unknown number of categories. A number of commercial pattern recognition systems are available for character recognition, signature recognition, document classification, fingerprint classification, speech and speaker recognition, white blood cell (leukocyte) classification, military target recognition, etc. Most machine (computer) vision systems employ pattern recognition techniques to identify objects for sorting, inspection, and assembly.
The design of a pattern recognition system consists of following main modules: (i) sensing, (ii) feature extraction, (iii) decision making, and (iv) performance evaluation. The availability of low cost and high resolution sensors (e.g., digital cameras, microphones and scanners) and data sharing over the Internet have resulted in huge repositories of digitized documents (text, speech, image and video). Need for efficient archiving and retrieval of this data has fostered the development of pattern recognition algorithms in new application domains (e.g., text, image and video retrieval, bioinformatics, and face recognition).
A pattern recognition system can be designed based on a number of different approaches: (i) template matching, (ii) geometric (statistical) methods, (iii) structural (syntactic) methods, and (iv) neural (deep) networks. This course will introduce the fundamentals of statistical pattern recognition with examples from several application areas. The course will cover techniques for visualizing and analyzing multi-dimensional data along with algorithms for projection, dimensionality reduction, clustering and classification. The course will present various approaches to classifier design so students can make judicious choices when confronted with real pattern recognition problems. It is important to emphasize that the design of a complete pattern recognition system for a specific application domain (e.g., remote sensing) requires domain knowledge, which is beyond the scope of this course. Students will use available MATLAB tools and will be expected to implement some algorithms using their choice of a programming language.
An undergraduate level understanding of probability, statistics and linear algebra is assumed. A basic knowledge of MATLAB and Python is essential.
|Jan 9||Introduction to Pattern Recognition (Ch 1)
Lecture-1 Slides: Pattern Recognition.
|Jan 11, 16, 18, 23||Statistical Decision Theory (Ch 2)
Lecture-2,3 Slides: Bayes Decision Theory
Jan 18: HW2 assigned, due on Jan 30
Notes on Neyman-Pearson decision rule
Notes on error rate of a linear discriminant function
|Jan 25, 30, Feb 1||Parameter Estimation (Ch 3)
Bayes Estimator for multivariate Gaussian density with unknown covariance matrices
Bayes Estimator under quadratic loss
Coin Tossing Example
Jan 30: HW3 assigned, due on Feb 13
|Feb 6||Curse of Dimensionality (Ch 3)
A Problem of Dimensionality: A Simple Example
|Feb 8, 13||Component analysis and Discriminants (Ch 3)
Principle Component Analysis (PCA)
PCA for face Recognition
Feb 13: HW4 assigned, due on March 1
|Feb 15, 20, 22||Neural Networks (Ch 6)
Discussion of Project
|Feb 27||Mid Term Exam|
Decision Trees (Ch 8)
HW5 assigned, due on Mar 15
|Mar 6, 8||SPRING BREAK|
|Mar 13, 15||
Non-parametric Techniques (Ch 4)
A Branch and Bound Algorithm for Computing k-Nearest Neighbors
Mar 15: HW6 assigned due on Mar 29
|Mar 22, 27||
Linear Discriminant functions (Ch 5)
Support Vector Machines
|Mar 29, Apr 3||Error Rate Estimation, Bagging, Boosting, Classifier Combination (Ch 9)
Mar 29: HW6 due
|Apr 5||Feature Selection
Feature Selection : Evaluation, Application, and Small Sample Performance by Jain and Zongker
April 5: Project Progress Report Due
|Apr 12, 17, 19|| Unsupervised Learning, Clustering, semi-supervised learning and Multidimensional Scaling (Ch 10)
Data Clustering : 50 Years Beyond K-means (Presentation Slides)
A Nonlinear Mapping for Data Structure Analysis by J. Sammon
A Global Geometric Framework for Nonlinear Dimensionality Reduction by J. Tanenbaum et al.
|Apr 24, 26||
April 23: Final Project Presentations Due 11:59 pm
Final Project Presentations
April 26: Final Project Report Due
|May 4||FINAL EXAM, 12:45pm - 2:45pm, 2245 EB|
The cumulative score will be mapped to the letter grade as follows: 90% or higher: 4.0; 85% to 90%: 3.5; 80% to 85%: 3.0 and so on.
Both the exams will be closed book. Makeup exams will be given ONLY if properly justified. Homework solutions must be turned in the class on the date they are due. Late homework solutions will not be accepted.
Article 2.3.3 of the Academic Freedom Report states "The student shares with the faculty the responsibility for maintaining the integrity of scholarship, grades, and professional standards." In addition, the Department of Computer Science and Engineering adheres to the policies on academic honesty as specified in General Student Regulations 1.0, Protection of Scholarship and Grades; the all-University Policy on Integrity of Scholarship and Grades; and Ordinance 17.00, Examinations. (See Spartan Life: Student Handbook and Resource Guide)
Therefore, unless authorized by your instructor, you are expected to complete all course assignments, including homework, projects, quizzes, tests and exams, without assistance from any source. You are expected to develop original work for this course; therefore, you may not submit course work you completed for another course to satisfy the requirements for this course. Students who violate MSU academic integrity rules may receive a penalty grade, including a failing grade on the assignment or in the course. Contact your instructor if you are unsure about the appropriateness of your course work. (See also the Academic Integrity webpage.)