Course Description
This course covers general issues of parallel and distributed
processing, which include system, algorithm, programming, performance
evaluation, and application of parallel computers.
Approximately two third of the course will be devoted to
basic concepts and techniques, and the remaining third will
be on programming and assorted current topics in parallel
processing, including cloud computing and GPGPU. We will not follow any particular text
through out the entire class. In stead, we will use two
texts as the general guideline of the lecture, one
for basic concepts and another one for programming
skill. Three reference books are also listed.
Assigned readings may give from time to time to complement the lectures.
Homework assignments will include question/answer execises and programming
assignments.
For the last part of the course we will read
an assortment of papers and work on different term projects.
This semester this class is uniquely taught jointly by three instructors, Prof. Sun from IIT, Dr. Thakur and Dr. Balaji from Angonne National Lab. Prof. Sun will cover the initial part of the class, overview, performance, and part of algorithm design and file systems. Dr. Thakur is a developer and an authority of MPI and MPI I/O. He will teach MPI and I/O system, and programming, including MPI programming and GPU programming. Dr. Balaji is an expert in high performance communication. He will cover all aspects of communication issues, from system development to algorithm design; and teach communiction related topics, such as Cloud and cluster computing.
Topics include:
- The Scope of Parallel Computing
- Parallel Architecture: Basic issues and its influence on programming
- Performance Evaluation
- Algorithm Design and Analysis: case study
- Parallel Programming: Pthread, OpenMP and MPI
- Network, communication, and distributed processing
- System Consideration: Partition, Communication, Scheduling and OS issues
- Data Access and I/O
- Cluster and Distributed Network Computing
- Other Current Trend of Parallel Processing
- See Detailed Class Schedule for more information regarding teaching schedule.
Course Materials
-
Recommended
- Ananth Grama, Anshul Gupta, and George Karypis, Vipin Kumar
- Introduction to Parallel Computing (2nd Edition)
- Addison-Wesley, 2003 (see here for additional resources related to the text).
- W. Gropp, et. al.,
- Using MPI Tow -Volume Set
- MIT Press, 1999 (see here for additional resources related to the text).
- B. Chapman, G. Jost and R. van der Pas,
- Using OpenMP: Portable Shared Memory Parallel Programming
- MIT Press, 2007 (see here for additional resources related to the text).
- Introduction to Parallel Computing (2nd Edition)
-
References
- Kai Hwang,Jack Dongarra, and Geoffrey C. Fox
- Distributed and Cloud Computing: From Parallel Processing to the Internet of Things
- Morgan Kaufmann Publishers, 2011 (see here for additional resources related to this text.)
- Ian Foster and Carl Kesselman
- The Grid 2: Blueprint for a New Computing Infrastructure
- Morgan Kaufmann Publishers, 2003 (see here for additional resources related to the text.)
- David Culler, Jaswinder Singh, and Anoop Gupta
- Parallel Computer Architecture: A hardware/software approach
- Morgan Kaufmann Publishers, 1999 (see here for additional resources related to the text.)
- Distributed and Cloud Computing: From Parallel Processing to the Internet of Things