CS595 Fall 2010: Probabilistic Graphical Models

Course Description

In this course, we will cover probabilistic graphical models: powerful and interpretable models for reasoning under uncertainty. We will survey a family of models, such as Bayesian networks, Markov networks, and factor graphs. The discussions will include both theoretical aspects of representation, learning, and inference in graphical models, and their applications to many interesting fields such as computer vision, natural language processing, computational biology, medical diagnosis, and more. Time permitting, we will also discuss specific graphical model representations such as Hidden Markov Models, Probabilistic Relational Models, Conditional Random Fields, etc.

Course Information

Time and Location: Tue - Thu 1:50 - 3:05pm in Stuart 225
Mustafa Bilgic
Office: Stuart 228C
Email: mbilgic AT iit.edu
Office Hours: Tue 3:30-4:30pm (Other times by appointment)

Course Rationale and Objectives

Probabilistic graphical models is one of the most exciting developments in machine learning. Here is a list of only a few reasons why this class might be useful and fun for you:


The course will be self-contained and thus there is no specific course requirement. However, familiarity with probability theory and graph theory is recommended (we will cover some of the basics in the class).

Course Format and Grading

The class will consist of a mixture of presentations and class discusions. As such, students are expected to read the required materials and participate in the discussions. Students are expected to return a short write-up on each reading by 11:59pm the day before the lecture. The purpose of this class is to learn about graphical models and apply them to research/application problems of your choice. You will be expected to carry-out a semester long project. The course will be fun, a worth-wile learning experience, and hopefully useful for your research problems.

Course Material

There is a required (see important note below) text book for this course:

Probabilistic Graphical Models, by Daphne Koller and Nir Friedman

There will be additional supplemental materials (mostly available on the web).

Important Note: If the cost of the textbook is preventing you from registering for the course, please contact me. A copy of the book is also expected to be available in the library reserves.

Reading Assignments

For each topic you will be expected to read a chapter and/or one to two papers. For each reading, you are expected to email me a write-up that includes:

Course Project

You are expected to carry-out a semester-long project. The project will be on a topic/problem of your choice. Ideally, it will help you formulate and solve a research problem that you are either already working on or considering to work on. You can use existing tools or write your own code or a combination. You are expected to:
More details on the project will be provided throughout the semester.

Course Mailing List

There is a course mailing list (pgm-f10@mailer). Announcements will be made to this list. Join using this link.

Audit Policy

If you are planning to audit the class without registering, please see me first. Because class participation is essential in this class, "passive" auditing is not allowed. Auditers are required to do the assignments and participate in the class discussions. Social pressure tactics will be used to enforce this rule.

Schedule (Will be adjusted based on the class interest)

The topics that will be covered and time spent on each topic will be adjusted based on the class interest and participation. The purpose of this class is not to "cover as much as possible"; rather, it is to "learn as much as possible." Thus, the following schedule is rather tentative.

Topic Notes and Readings
Course logistics
Assignment 1 due 8/25, 11:59pm

Chapter 1

Background Material
Chapter 2
Assignment 2 due 9/1, 6:00pm

9/2, 9/7, 9/9
Representation - Bayesian networks
Chapter 3
Assignment 3, due 9/8, 6:00pm
Hugin file for the student example (Figure 3.4)

Representation - CPDs

Chapter 5

9/16, 9/21
Representation - Markov networks

Chapter 4, Assignment 4, due 9/15, 6:00pm

Application showdown

Assignment 5, due 9/22, 6:00pm

9/28, 9/30

Inference - Variable elimination

Sections 9.1, 9.2, and 9.3.
Assignment 6, due 9/29, 6pm.

10/5, 7, 12
Inference - Message passing

Sections 10.1 and 10.2.

Assignment 7, due in class on 10/5.

Inference - MAP Inference

Sections 13.1, 13.2, 13.3
Assignment 8, due 6pm, Wed, 10/13
Possible Toolkits

Inference - Sampling

Sections 12.1, 12.2, 12.3
Project Proposal, due in class, 10/19

Inference, Algorithm evaluation

Proposal comments, my research area

Project proposal comments due

Learning - Overview

Chapter 16

Learning - Bayesian Networks

Assignment 9, due 6pm Wed, 11/03


Learning - Markov Networks


Template-based representations


Template-based representations


Project presentations