CS 451 Introduction to Parallel and Distributed Computing (Spring 2026)

Instructor: Dr Nik Sultana, Department of Computer Science, Illinois Institute of Technology
Lecture Time: TR 10:00-11:15 CT    Lecture Location: SB 104
TA: Alexander Wolosewicz (awolosewicz@hawk).
Office Hours:     Alexander: Tuesday and Thursday at 14:00-15:00,       Nik: Tuesdays at 09:00-10:00
Announcements: On Canvas.
Course content: This page and Canvas.
Grading: Assignments (30%), Project (5% for Project design, 10% for Progress Update, and 15% for final submission), Mid-term Exam (20%), Final Exam (20%).
Policies: See the policies page.
Resources:

Course Material

Screen grab of an LLM running on FABRIC using one of the GPUs on FABRIC.
This example shows an LLM being used on FABRIC, using one of FABRIC's GPUs. This introductory course revolves around practical and up-to-date examples of applications of parallel and distributed computing. These examples include both common and advanced applications. The example shown above cuts across several topics in the course. We use that example to compare token throughput when using a GPU and when using different kinds of parallelism that are provided by a modern CPU.
These are the course notes:
  1. Introduction to Parallel and Distributed Computing
  2. Software Compartmentalization
  3. Parallel and Distributed Computing Concepts
  4. Program-level Abstractions for Parallelism
  5. OpenMP
  6. Low-level programming
  7. Vector processing
  8. Message-Passing Interface (MPI)
  9. CUDA
  10. SystemVerilog
Various FABRIC resources were developed for this course too—to set up and run the LLM example, MPI, etc.

Course Calendar

   The final exam takes place during Exam Week and isn't shown in the course calendar. The date of that exam is set by IIT's Registrar and you can see their full exam schedule. For this course, the final exam takes place at 2pm on May 5 in SB104.

MondayTuesday WednesdayThursday Friday
Jan 12
Jan 13
LEC 1: Introduction and Course Outline
Jan 14
Jan 15
LEC 2: Introduction to FABRIC
Jan 16
Assignment 1 released
Jan 19
MLK day -- No Classes
Jan 20
LEC 3: Software Compartmentalization
Jan 21
Jan 22
LEC 4: Software Compartmentalization
Jan 23
DUE: Assignment 1
Assignment 2 released
Jan 26
Jan 27
LEC 5: Software Compartmentalization
Jan 28
Jan 29
LEC 6: Software Compartmentalization
Jan 30
Feb 02
Feb 03
LEC 7: Review of concepts
Feb 04
Feb 05
LEC 8: Multi-threading
Feb 06
DUE: Assignment 2
Assignment 3 released
Feb 09
Feb 10
LEC 9: Synchronization
Feb 11
Feb 12
LEC 10: Guest lecture: Alexander Wolosewicz:
Precise timing in distributed measurements
Based on this paper from INDIS'25.
Feb 13
Feb 16
Feb 17
LEC 11: IPC
Feb 18
Feb 19
LEC 12: IPC
Feb 20
DUE: Assignment 3
Assignment 4 released
Feb 23
Feb 24
Mid-term Exam
Feb 25
DUE: Project Design
Feb 26
LEC 13: OpenMP
Feb 27
Mar 02
Mar 03
LEC 14: OpenMP
Mar 04
Mar 05
LEC 15: OpenMP
Mar 06
DUE: Assignment 4
Assignment 5 released
Mar 09
Mar 10
LEC 16: OpenMP
Mar 11
Mar 12
LEC 17: Review: low-level programming
Mar 13
Mar 16
Spring Break Week -- No Classes
Mar 17
Spring Break Week -- No Classes
Mar 18
Spring Break Week -- No Classes
Mar 19
Spring Break Week -- No Classes
Mar 20
Spring Break Week -- No Classes
Mar 23
Mar 24
LEC 18: Review: low-level programming
Mar 25
Mar 26
LEC 19: SIMD
Mar 27
DUE: Assignment 5
Mar 30
DUE: Project Progress Update
Mar 31
LEC 20: Guest lecture: Kevin A. Brown, Argonne National Lab
"Understanding and Predicting Network Performance: It’s Hard"
Apr 01
Apr 02
LEC 21: SIMD
Apr 03
Apr 06
Apr 07
LEC 22: MPI
Apr 08
Apr 09
LEC 23: MPI
Apr 10
Apr 13
Apr 14
LEC 24: CUDA
Apr 15
Apr 16
LEC 25: CUDA
Apr 17
Apr 20
DUE: Project (Final Submission)
Apr 21
LEC 26: CUDA
Apr 22
Apr 23
LEC 27: SystemVerilog
Apr 24
Apr 27
Apr 28
LEC 28: SystemVerilog
Apr 29
Apr 30
LEC 29: SystemVerilog
May 01