[Syllabus] EECS 595: Natural Language Processing
EECS 595: Natural Language Processing
Instructor: Joyce Chai
Course Description
The field of Natural Language Processing (NLP) is primarily concerned with computational models and computer algorithms to process human languages, for example, automatically interpret, generate, and learn natural language. In the past twenty years, the rise of the world wide web and social media have created tremendous opportunities for exciting NLP techniques and applications. The advances in deep learning have also paved the way to create large-scale language models and tackle many NLP problems in the real world. This course provides an introduction to the state of the art in NLP including large language models, syntax, semantics, discourse, and their applications in information extraction, question answering, and converstional systems.
Text book for Reference
- Speech and Language Processing, an introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, third edition (draft), by Daniel Jurafsky and James Martin, Prentice Hall
- Neural Network Methods for Natural Language Processing, Yoav Goldberg, Synthesis Lectures on Human Language Technologies (optional)
- Natural Language Processing with Transformers, by Lewis Tunstall, Leandro von Werra, and Thomas Wolf, O’Reilly (optional)
Prerequisite
- Proficiency in Python programming (using NumPy and PyTorch).
- Knowledge and experience in machine learning.
Schedule of Topics
| Topics |
|---|
| Introduction |
| Text Classification, Logistic Regression |
| Neural Networks and Backpropagation |
| Word Vectors and Vector Semantics |
| Language Modeling |
| Recurrent Neural Networks |
| Sequence-to-Sequence Models |
| Transformers |
| Pre-trained Large Language Models |
| Fine-tuning and Prompting |
| Constituency Parsing |
| Dependency Parsing |
| Meaning Representations |
| Semantic Parsing |
| Semantic Roles and Selectional Restriction |
| Coreference Resolution |
| Discourse Coherence |
| Information Extraction |
| Question Answering |
| Commonsense Reasoning |
| Conversational Systems |
| Advanced Topics |
Course Policies
Course Grades
The work in this course consists of four homework assignments and a final project. Each assignment may include a written portion and a programming portion. All homework must be your own work.
Late submission policy
You have up to 3 days after the due date to submit your assignments. After that cut off date, you will receive 0 point. For each day delayed, you will receive a penalty for the assignment. If there is a special circumstance, please contact the instructor/TAs directly.
Academic Honesty
All homework assignments submitted must be your own work. Review the college of Engineering’s Honor Code here: http://www.engin.umich.edu/college/academics/bulletin/rules (Links to an external site.)
Special Accommodations
If you have disabilities or medical conditions which require some form of accommodations, please make an appointment with the instructor within the first week of classes.
Notes: The instructor reserves the right to modify course policies and the course calendar according to the progress and needs of the class.