Ioan Raicu

Illinois Institute of Technology

Argonne National Laboratory

CS451: Introduction to Parallel and Distributed Computing

Semester: Spring 2014

Lecture Time: Tuesday/Thursday, 11:25AM-12:40PM

Lecture Location: Stuart Building 104

Professor: Ioan Raicu

Office Hours Time: Thursday 2PM-3PM

Office Hours Location: Stuart Building 237D

Teaching Assistant: Tonglin Li

Office Hours Time: Thursday 10AM-11AM, Friday 12:45PM-1:45PM

Office Hours Location: SB002

Teaching Assistant: Iman Sadooghi

Office Hours Time: Monday/Wednesday 12:45PM-1:45PM

Office Hours Location: SB003b



For those of you working towards the Master Of Computer Science With a Specialization in Distributed and Cloud Computing , we know how important CS553 is for your coursework towards satisfying the necesary requiremetns towards your degree. Since we are not teaching CS553 in the Spring 2014 (as expected), we have added CS451 to the list of potential courses satisfying the needed requirements of the specialization. Please contact Ioan Raicu at if you have any questions about this.


Course Overview

This course covers general introductory concepts in the design and implementation of parallel and distributed systems, covering all the major branches such as Cloud Computing, Grid Computing, Cluster Computing, Supercomputing, and Many-core Computing. The specific topics that this course will cover are:  asynchronous/synchronous computation/communication, concurrency control, fault tolerance, GPU architecture and programming, heterogeneity, interconnection topologies, load balancing, memory consistency model, memory hierarchies, Message passing interface (MPI), MIMD/SIMD, multithreaded programming, parallel algorithms & architectures, parallel I/O, performance analysis and tuning, power, programming models (data parallel, task parallel, process-centric, shared/distributed memory), scalability and performance studies, scheduling, storage systems, and synchronization. Some of these topics are covered in more depth in the graduate courses focusing on specific sub-domains of distributed systems, such CS546, CS550, CS553, CS554, CS570, and CS595. While this CS451 course is not a pre-requisite to any of the graduate level courses in distributed systems, both undergraduate and graduate students who wish to be better prepared for these courses could take this CS451 course. This course involves lectures, programming assignments, and exams. This course was offered as CS495 in the past. Prerequsites: CS351 or CS450. 

You can find the detailed syllabus here. More details will be posted here soon.


Course Overview

We have setup a mailing list at Please post any questions you may have there.



Slides for all lectures are posted on BB.

Date Description Reading Assignment
01-14-2014 Introductions Syllabus  
01-16-2014 Introduction to Distributed Systems   Project #1 out (Benchmarks)
Sample Code (ZIP, TAR)
01-21-2014 Introduction to Distributed Systems   Quiz #1
SB204, 12:45PM-1:45PM
Tutorial on Linux and Project #1    
01-23-2014 Introduction to Distributed Systems    
Class runs from 12PM-12:40PM
Introduction to Distributed Systems   Quiz #2
01-30-2014  Distributed System Models  and Enabling Technologies Chapter 1  
02-04-2014 Distributed System Models  and Enabling Technologies Chapter 1 Proj #1 due
Quiz #3
02-06-2014 Distributed System Models  and Enabling Technologies Chapter 1  
02-11-2014 Distributed System Models  and Enabling Technologies Chapter 1 Quiz #4
02-13-2014 Distributed System Models  and Enabling Technologies Chapter 1 Proj #2 out (GPUs)
02-15-2014 Tutorial on Jarvis & CUDA    
02-18-2014 Multi/Many-Core Computing 1. Amdahl's law in the multicore era (PDF)
2. Reevaluating Amdahl's Law in the Multicore Era (PDF)
Quiz #5
02-20-2014 Memory System Parallelism for Data –Intensive  and Data-Driven Applications

Guest Lecture, Dr. Xian-He Sun
1. Concurrent Average Memory Access Time (PDF)
2. Pattern-Direct and Layout-Aware Replication Scheme for Parallel I/O Systems (PDF)
02-25-2014 Multi/Many-Core Computing   Quiz #6
02-27-2014 System Architectures   Proj #2 due
03-04-2014 Midterm Exam (75 minutes)   Proj #3 out (P2P)
03-06-2014 System Architectures    
03-11-2014 System Architectures   Quiz #7
03-13-2014 Parallel Programming    
03-18-2014 Spring Break (NO CLASS)    
03-20-2014 Spring Break (NO CLASS)    
03-25-2014 Parallel Programming   Proj #3 due
Proj #4 out (MPI)
Quiz #8
03-27-2014 Parallel Programming    
04-01-2014 Parallel Programming   Quiz #9
04-03-2014 MapReduce    
04-08-2014 MapReduce   Quiz #10
04-10-2014 MapReduce   Proj #4 due
Proj #5 out (MapReduce/Amazon)
04-15-2014 Cloud Computing   Quiz #11
04-17-2014 Cloud Computing    
04-22-2014 Cloud Computing   Quiz #12
04-24-2014 Cloud Computing    
04-29-2014 Cloud Computing   Proj #5 due
05-01-2014 Final Exam (75 minutes)    





Next Semester Fall 2014

CS553: Cloud Computing