Pattern recognition techniques are used to automatically classify physical objects (handwritten characters, tissue samples, faces) or abstract multidimensional patterns (n points in d dimensions) into known or possibly unknown number of categories. A number of commercial pattern recognition systems are available for character recognition, signature recognition, document classification, fingerprint classification, speech and speaker recognition, white blood cell (leukocyte) classification, military target recognition, etc. Most machine (computer) vision systems employ pattern recognition techniques to identify objects for sorting, inspection, and assembly.
The design of a pattern recognition system consists of following main modules: (i) sensing, (ii) feature extraction, (iii) decision making, and (iv) performance evaluation. The availability of low cost and high resolution sensors (e.g., digital cameras, microphones and scanners) and data sharing over the Internet have resulted in huge repositories of digitized documents (text, speech, image and video). Need for efficient archiving and retrieval of this data has fostered the development of pattern recognition algorithms in new application domains (e.g., text, image and video retrieval, bioinformatics, and face recognition).
A pattern recognition system can be designed based on a number of different approaches: (i) template matching, (ii) geometric (statistical) methods, (iii) structural (syntactic) methods, and (iv) neural (deep) networks. This course will introduce the fundamentals of statistical pattern recognition with examples from several application areas. The course will cover techniques for visualizing and analyzing multi-dimensional data along with algorithms for projection, dimensionality reduction, clustering and classification. The course will present various approaches to classifier design so students can make judicious choices when confronted with real pattern recognition problems. It is important to emphasize that the design of a complete pattern recognition system for a specific application domain (e.g., remote sensing) requires domain knowledge, which is beyond the scope of this course. Students will use available MATLAB tools and will be expected to implement some algorithms using their choice of a programming language.
An undergraduate level understanding of probability, statistics and linear algebra is assumed. A basic knowledge of MATLAB and Python is essential.
| Jan 9 | Introduction to Pattern Recognition (Ch 1) |
| Jan 11, 18, 23, 25 | Statistical Decision Theory (Ch 2, 2.1 to 2.10)
Jan 18: HW2 assigned, due on Feb 1 Notes on Neyman-Pearson decision rule Notes on error rate of a linear discriminant function |
| Jan 30, Feb 1 | Parameter Estimation (Ch 3, 3.1 to 3.5, 3.7 to 3.8)
Bayes Estimator for multivariate Gaussian density with unknown covariance matrices |
| Feb 6 | Component analysis and Discriminants (Ch 3)
Principle Component Analysis (PCA) PCA for face Recognition |
| Feb 8, 13, 15 | Neural Networks (Ch 6, 6.1 to 6.3) (Lecture - 9 Slides)
Deep Networks (Lecture - 10, 11 Slides) Notes on Deep Networks Discussion of Project |
| Feb 20 | Non-parametric Techniques (Ch 4, 4.1 to 4.3.4) |
| Feb 22 | Non-parametric Techniques (Ch 4, 4.4 to 4.6.1) (Lecture - 12, 13 Slides) |
| Feb 27 | Review of Mid Term Exam |
| Mar 1 | Mid Term Exam |
|---|---|
| Mar 6, 8 | SPRING BREAK |
| Mar 13, 15 |
Curse of Dimensionality (Ch 3.7) (Lecture - 14 Slides) A Problem of Dimensionality: A Simple Example Feature Selection (Lecture - 15 Slides) Feature Selection : Evaluation, Application, and Small Sample Performance by Jain and Zongker |
| Mar 22, 27 |
Decision Trees (Ch 8, 8.1 to 8.3) (Lecture - 16 Slides) Support Vector Machines (Lecture - 17 Slides) Mar 24: HW5 assigned, due on Apr 10 March 27: Project Progress Report Due |
| Mar 29, Apr 3 | Error Rate Estimation, Bagging, Boosting, Classifier Combination (Ch 9)
Logistic Regression (Lecture - 18 Slides) Clustering (Lecture - 19-20 Slides) |
| Apr 5, 10 |
Unsupervised and semi-supervised learning (Ch. 10) Apr 10: HW5 due |
| Apr 12 | EXAM-II |
| Apr 17, 19 | Clustering and Multidimensional Scaling (Ch 10) (Lecture - 21-22 Slides)
Data Clustering : 50 Years Beyond K-means (Presentation Slides) Graph Theoretical Methods for Detecting and Describing Gestalt Clusters by C. Zahn A Nonlinear Mapping for Data Structure Analysis by J. Sammon A Global Geometric Framework for Nonlinear Dimensionality Reduction by J. Tanenbaum et al. |
| Apr 24, 26 |
April 23: Final Project Presentations Due 11:59 pm Final Project Presentations |
| May 1 | Final Project Report Due |
The cumulative score will be mapped to the letter grade as follows: 90% or higher: 4.0; 85% to 90%: 3.5; 80% to 85%: 3.0 and so on.
Both the exams will be closed book. Makeup exams will be given ONLY if properly justified. Homework solutions must be turned in the class on the date they are due. Late homework solutions will not be accepted.
Article 2.3.3 of the Academic Freedom Report states "The student shares with the faculty the responsibility for maintaining the integrity of scholarship, grades, and professional standards." In addition, the Department of Computer Science and Engineering adheres to the policies on academic honesty as specified in General Student Regulations 1.0, Protection of Scholarship and Grades; the all-University Policy on Integrity of Scholarship and Grades; and Ordinance 17.00, Examinations. (See Spartan Life: Student Handbook and Resource Guide)
Therefore, unless authorized by your instructor, you are expected to complete all course assignments, including homework, projects, quizzes, tests and exams, without assistance from any source. You are expected to develop original work for this course; therefore, you may not submit course work you completed for another course to satisfy the requirements for this course. Students who violate MSU academic integrity rules may receive a penalty grade, including a failing grade on the assignment or in the course. Contact your instructor if you are unsure about the appropriateness of your course work. (See also the Academic Integrity webpage.)