CEng 783 - Deep Learning - Fall 2017

[Image generated by http://deepdreamgenerator.com/]

Instructor: Asst. Prof. Dr. Emre Akbas; e-mail: emre at ceng dot metu.edu.tr; Office: B-202; Office hours by apppointment
Teaching Assistant: Ezgi Ekiz is kindly volunteering. Thank you, Ezgi.
Lectures: Fridays 13:40-16:30 at BMB-5 ~~BMB-3~~
Online communication: (e-mail list, forum, homework submissions) https://odtuclass.metu.edu.tr/
Syllabus: pdf

Announcements

18/12/17: Place change: from now on, our class will be in BMB5.
11/11/17: Midterm exam will be held in class next week (Nov 17, 13:40 @ BMB-3)
3/11/17: Assignment 2 submission postponed 1 week. New due date is 13/11 (submit via ODTUClass)
6/10/17: Students with these IDs can add the class during the add/drop period next week.
27/09/17: Those who want to register to the class but could not due to the limited capacity should come to the first lecture.

Late submission policy

Any work, e.g. an assignment solution, that is submitted late past its deadline will receive -10 points per day delayed. For example, if the deadline is Oct 14, 23:55 and a student submits his/her work anytime on Oct 15, that work will be evaluated over 90 instead 100.

Detailed syllabus:

An important note: Lecture slides provided below are by no means a “complete” resource for studying. I frequently use the board in class to supplement the material in the slides.

NOTE: Below, links to slides and homework material are broken because I removed the files. Recent versions of these files can be found in the Fall 2019 version of the course.

Week 1

Lecture topics: A high-level introduction to Deep Learning (slides.) Misc info about the course.
Reading assignment for next week: “Chapter 5: Machine Learning Basics” from the book “Deep Learning.”
Assignment 1.

Week 2

Lecture topics: Machine learning background and basics (slides.)
Ipython notebook for the in-class hands-on demo on binary classification, gradient descent, hinge loss: download

Week 3

Lecture topics: Biological neuron, artificial neuron, Perceptron, Multilayer Perceptrons, Artificial Neural Networks, Backpropagation, Activation Functions, Stochastic Gradient Descent, Momentum (slides.)
Adding regularization to the hinge loss classifier download
Reading assignment for next week: “Chapter 8: Optimization for trainig deep models” from the book “Deep Learning.”

Week 4

Lecture topics: Convolutional neural networks, convolution, connectivity types, pooling, AlexNet (slides.)
Hw2 and project proposal template are announced at ODTUClass.
Reading assignment for next week: LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.

Week 5

Lecture topics: Convolutional neural networks continued, multiclass hinge loss, derivation of cross-entropy loss, notes on implementing backpropagation in a modular way, variants of stochastic gradient methods, adaptive learning rate methods (slides.)
Assignment 2 was announced. It is on implementing a modular backpropagation network and setting up, training, testing of ConvNets in Tensorflow. Download it from ODTUclass.
Reading assignment for next week:
- He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.
- Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems (pp. 91-99).

Week 6

Lecture topics: ConvNet initialization, batch normalization, fine tuning. Some applications of ConvNets. (slides)
Reading assignment for next week: A. Karpathy’s blog post on recurrent neural networks

Week 7

No lecture. Midterm exam.

Week 8

Midterm exam solutions.
Lecture topics: Recurrent Neural Networks, LSTM. (slides)

Week 9

Lecture topics: RNN recap, LSTM recap, some applications of RNNs. (slides)

Week 10

Lecture topics: Deep Generative Models (Unsupervised Learning). (slides)

Week 11

Project progress demos/presentations in class.

Week 12

Lecture topics: Deep Reinforcement Learning. (slides)
Hands-on RNN (LSTM) tutorial: jupyter notebook, standalone Python code.

Week 13

Lecture topics: Some notes on deep hierarchies in human/biological vision. (slides)

Week 14

Project final presentations in class. Groups expected to present are announced at ODTUClass.

Content will be added here as we go…