CS 595: Probabilistic Graphical Models

Course Description

In this course, we will cover probabilistic graphical models: powerful and interpretable models for reasoning under uncertainty. We will survey a family of models, such as Bayesian networks, Markov networks, and factor graphs. The discussions will include both theoretical aspects of representation, learning, and inference in graphical models, and their applications to many interesting fields such as computer vision, natural language processing, computational biology, medical diagnosis, and more. Time permitting, we will also discuss specific graphical model representations such as Hidden Markov Models, Probabilistic Relational Models, Conditional Random Fields, etc.

Course Information

Time and Location: Tue - Thu 1:50 - 3:05pm in Stuart 225
Professor: Mustafa Bilgic
Office: Stuart 228C
Email: mbilgic AT iit.edu
Office Hours: Tue 3:30-4:30pm (Other times by appointment)

Course Rationale and Objectives

Probabilistic graphical models is one of the most exciting developments in machine learning. Here is a list of only a few reasons why this class might be useful and fun for you:

Probabilistic graphical models are used everywhere - speech recognition, sensor network data modeling, computer vision, computational biology, planning under uncertainty, structured data analysis, natural language processing, and many many more.
Course will have an application component - helps you model your research problems using graphical models, which provides new perspectives and insights
You will be familiar with terms and models used in an increasing number of papers - Bayesian networks, Markov networks, Conditonal random fields, plate models, Gibbs sampling, belief propagation, etc.
Experience with probabilistic graphical models is always good on your CV/resume.

Prerequisites

The course will be self-contained and thus there is no specific course requirement. However, familiarity with probability theory and graph theory is recommended (we will cover some of the basics in the class).

Course Format and Grading

The class will consist of a mixture of presentations and class discusions. As such, students are expected to read the required materials and participate in the discussions. Students are expected to return a short write-up on each reading by 11:59pm the day before the lecture. The purpose of this class is to learn about graphical models and apply them to research/application problems of your choice. You will be expected to carry-out a semester long project. The course will be fun, a worth-wile learning experience, and hopefully useful for your research problems.

Class participation: 30%
Reading assignments: 30%
Project: 40%
Exams: There will not be any exam in this class.

Course Material

There is a required (see important note below) text book for this course:

Probabilistic Graphical Models, by Daphne Koller and Nir Friedman

There will be additional supplemental materials (mostly available on the web).

Important Note: If the cost of the textbook is preventing you from registering for the course, please contact me. A copy of the book is also expected to be available in the library reserves.

Reading Assignments

For each topic you will be expected to read a chapter and/or one to two papers. For each reading, you are expected to email me a write-up that includes:

General comments on the topic (3-5 sentences)
Two-three questions that you would like to be answered in class
Answer an assigned question (3-5 sentences)

Course Project

You are expected to carry-out a semester-long project. The project will be on a topic/problem of your choice. Ideally, it will help you formulate and solve a research problem that you are either already working on or considering to work on. You can use existing tools or write your own code or a combination. You are expected to:

Write a project proposal (1-2 pages)
Present your project in class at the end of the semester
Write a final report (6-8 pages)

More details on the project will be provided throughout the semester.

Course Mailing List

There is a course mailing list (pgm-f10@mailer). Announcements will be made to this list. Join using this link.

Audit Policy

If you are planning to audit the class without registering, please see me first. Because class participation is essential in this class, "passive" auditing is not allowed. Auditers are required to do the assignments and participate in the class discussions. Social pressure tactics will be used to enforce this rule.

Schedule (Will be adjusted based on the class interest)

The topics that will be covered and time spent on each topic will be adjusted based on the class interest and participation. The purpose of this class is not to "cover as much as possible"; rather, it is to "learn as much as possible." Thus, the following schedule is rather tentative.

Date	Topic	Notes and Readings
8/24	Course logistics	Syllabus Assignment 1 due 8/25, 11:59pm
8/26	Introduction	Chapter 1
8/31	Background Material	Chapter 2 Assignment 2 due 9/1, 6:00pm
9/2, 9/7, 9/9	Representation - Bayesian networks	Chapter 3 Assignment 3, due 9/8, 6:00pm Hugin file for the student example (Figure 3.4)
9/14	Representation - CPDs	Chapter 5
9/16, 9/21	Representation - Markov networks	Chapter 4, Assignment 4, due 9/15, 6:00pm
9/23	Application showdown	Assignment 5, due 9/22, 6:00pm
9/28, 9/30	Inference - Variable elimination	Sections 9.1, 9.2, and 9.3. Assignment 6, due 9/29, 6pm.
10/5, 7, 12	Inference - Message passing	Sections 10.1 and 10.2. Assignment 7, due in class on 10/5.
10/14	Inference - MAP Inference	Sections 13.1, 13.2, 13.3 Assignment 8, due 6pm, Wed, 10/13 Possible Toolkits
10/19	Inference - Sampling	Sections 12.1, 12.2, 12.3 Project Proposal, due in class, 10/19
10/21	Inference, Algorithm evaluation
10/26	Proposal comments, my research area	Project proposal comments due
10/28	Learning - Overview	Chapter 16
11/02	Learning - Bayesian Networks	Assignment 9, due 6pm Wed, 11/03
11/09	Learning - Markov Networks
11/16	Template-based representations
11/23	Template-based representations
11/30	Project presentations

CS595 Fall 2010: Probabilistic Graphical Models