This course provides an introduction to foundations and the state-of-the-art technology enabling natural language communication with artificial agents. Topics include speech recognition, acoustic modeling and language modeling, dialogue and discourse modeling,  psycholinguistic studies on situated human language processing, and their applications in situated human robot dialogue.  These topics will be examined through reading, discussion, and hands-on experience with situated conversational systems.

 

Text book:

There is no required textbook for this course. The following books are reserved in the Engineering Library for your reference: 

• Speech and Language Processing by Jurafsky and Martin, second edition, Prentice Hall. (JM)
• Spoken Language Processing: A Guide to Theory, Algorithm and System Development by X. Huang, A. Acero, H. Hon, and R. Reddy.  Prentice-Hall, 2001. (Huang et al.)
• Approaches to Studying World-Situated Language Use,  Edited by John C. Trueswell and Michael K. Tanenhaus, MIT Press. (TT)
• Reinforcement Learning, by Richard Sutton and Andrew Barto, The MIT Press.

Grading

The final grade is determined based on: class participation and discussion,  written and programming assignments, paper presentations, and a final project.

Syllabus

Week	Class Date	Topic	Due
1	Jan 9	Introduction JM: Chapter 7.1-7.3
	Jan 11	Hidden Markov Model Lawrence R. Rabiner, 1989. A tutorial on hidden Markov models and selected applications in speech recognition, Proceedings of the IEEE 77(2), pp. 257-286. Errata by Ali Rahimi JM: Chapter 6.1-6.5
2	Jan 16	No Class - Martin Luther King Day
	Jan 18	Hidden Markov Model (2)	Homework 1 assigned
3	Jan 23	Search and Decoding in HMM for ASR JM: Chapter 9.6
	Jan 25	Acoustic Modeling JM: Chapter 9.3-9.4
4	Jan 30	Acoustic Modeling (2), Huang et al., P394-396
	Feb 1	Language Modeling JM: Chapter 4 S. Chen and J. Goodman, An Empirical Study of Smoothing Techniques for Language Modeling   Gale and Sampson, Good-Turing Frequency Estimation without Tears S. Katz, Estimation of probabilities from sparse data for the language model component of a speech recogniser.
5	Feb 6	No Class	Homework 1 due Homework 2 assigned
	Feb 8	Evaluation, Higher order HMM in decoding L. Gillick and S. Cox, Some Statistical Issues in the Comparison of Speech Recognition Algorithms,
6	Feb 13	Multi-pass decoding, A*decoding JM: Chapter 10 Huang et al., Chapter 12.5
	Feb 15	Some introduction to graph-based approaches, Changsong
7	Feb 20	More about graph-based approaches, Rui
	Feb 22	Semantic Processing of Natural Language
8	Feb 27	Semantic Processing of Spoken Language in Situated Interaction P. Gorniak and D. Roy (2004). Grounded Semantic Composition for Visual Scenes, Journal of Artificial Intelligence Research, 21: 429-470.
	Feb 29	Introduction to Dialogue Systems  (1)	Homework 2 due
9	Mar 5	No Class - Spring Break
	Mar 7	No Class - Spring Break
10	Mar 12	Dialogue Acts
	Mar 14	Paper Presentations: Couri	Project proposal due
11	Mar 19	Plan-based Dialogue Management
	Mar 21	Markov Decision Process (MDP)
12	Mar 26	POMDP
	Mar 28	Reinforcement Learning
13	Apr 2	RL in Dialogue Management Satinder S. et al., Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System. Journal of Artificial Intelligence Research, 2002.
	Apr 4	Paper Presentations: Deng	Project progress report due
14	Apr 9	Paper Presentations:  Meredith
	Apr 11	Paper Presentations: Lanbo
15	Apr 16	Paper Presentations: Zheyun
	Apr 18	Final Project Presentation
16	Apr 23	Final Project  Presentation
	Apr 25	Final Project Presentation	Project final report due: May 1

Academic Honesty

It is your responsibility to follow MSU's policy on academic integrity. Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment in which the copying or paraphrasing was done. Violation of academic integrity policy will result in a Grade F in the course. 

Alternative Testing:

Alternative testing is available to those with a documented disability affecting performance on tests. Students with documented disabilities requiring some form of accommodation receive a Verified Individualized Services and Accommodations (VISA) document which displays verified testing accommodations when appropriate. Please visit Alternative Testing Guidelines if applied. 

Notes: The instructor reserves the right to modify course policies and the course calendar according to the progress and needs of the class.