CS 577
Deep Learning (Spring 2020)

Course outline

What to expect from this course

Deep learning can be covered at different levels. The focus of this course is principles, mathematical concepts, algorithms, and techniques used in deep learning. Students in the course are expected to write computer programs implementing different techniques taught in the course. The course requires mathematical background and some programming experience. This course does not intend to teach how to use a specific application software but will use a GPU frameworks for the assignments.

Objectives

1. Provide overview of machine learning and context for the sub-area of deep learning.
2. Introduce fundamental problems in deep learning.
   
3. Provide understanding of theoretical foundations, techniques, mathematical concepts, common architectures, and algorithms used in deep learning and the way to apply them to solve problems.
4. Provide understanding of computational methods for deep learning and practice design and implementation of  deep learning using GPU frameworks. 
5. Provide pointers into the literature and exercise a project based on a literature search and one or more research papers to facilitate further study in this area.
   
   

Overview

1. Introduction to machine learning and computational foundations
2. Introduction to GPU frameworks
3. Neural networks
4. Deep feedforward networks
5. Regularization and optimization
6. Convolutional networks
7. Representation Learning
8. Sequence models and recurrent networks
9. Generative models
10. Deep reinforcement learning
11. Advanced architectures
12. Applications

Grading

1. There is an additional mandatory assignment (assignment 0) which does not carry any credit. There is a penalty of 5% for not submitting this assignment.
   
2. A certain percentage of the students may be invited to discuss their assignments.
3. Late days: there is a total of 6 "free late days" with no grade penalty for all the assignments to cover various reasons such as not feeling well, being busy, etc. Up to 2 free late days may be applied to each assignment. Being late beyond what is allowed by the free late days will result in a grade reduction of 25% per day. Late days are counted past midnight when an assignment is due and include weekends and holidays. The final project can not be late and no submissions will be accepted beyond the last day of class.
4. Each member of this course bears responsibility for maintaining the highest standards of academic integrity. All breaches of academic integrity must be reported immediately. Copying of programs from any source (e.g. other students or the web) is considered to be a serious breach of academic integrity.
5. The usual grade scale applies: A  > 90, B > 80, C > 70, etc.
6. If you cannot attend a scheduled exam (or project presentation) whether for justified or unjustified reason you must notify me by email. Not attending an exam (or project presentation) without prior notification will result in failing the exam (or project presentation) whether the reason was justified or not. Claiming not to know the date of an exam (or project presentation) is not a valid reason and will result in failing the exam (or project presentation).
   

Books

1. Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT Press, 2016.
2. Neural Networks and Deep Learning: A Textbook, Charu C. Aggarwal, Springer 2018.
3. Deep Learning with Python, Francois Chollet, Manning 2017.
   

Tentative schedule

class	date	topic	assignment
1	01/13	Introduction	AS0
2	01/15
3	01/20	No class (MLK day)
4	01/22	Introduction to GPU frameworks
5	01/27		AS1
6	01/29	Neural networks
7	02/03
8	02/05
9	02/10	Deep feedforward networks
10	02/12		AS2
11	02/17
12	02/19	Regularization and optimization
13	02/24
14	02/26
15	03/02	Convolutional networks	PROJ
16	03/04	Midterm
17	03/09
18	03/11		AS3
19	03/16	Spring break
20	03/18	Spring break
21	03/23	Representation learning
22	03/25		AS4
23	03/30	Recurrent networks
24	04/01
25	04/06	Generative models
26	04/08		AS5
27	04/13	Deep reinforcement learning
28	04/15
29	04/20	Presentations
30	04/22
31	04/27
32	04/29
33	05/04	Final exam 5-7pm