CENG 783 - Deep Learning - Spring 2017
Introduction to Machine Learning; Deep hierarchies and learning mechanisms in humans; Artificial neural networks; Deep vs. shallow architectures; Representation in terms of basis functions; Representation learning; Independent component analysis; Sparse representations; Convolutional neural networks; Restricted Boltzmann Machines; Deep Belief networks; Applications to pattern recognition, speech recognition and natural language processing.
You can access the syllabus here.
HW3. Deadline: June 4, 2017.
HW2. Deadline: May 7, 2017.
HW1. Deadline: March 20, 2017.
- Week 1: Introduction to Deep Learning, Machine Learning and Optimization.
Slide Set 1: Course overview, Introduction to Deep Learning
Slide Set 2: Introduction to Machine Learning and Optimization (we will continue with this set in the second week)
- Week 2: Continue on our introduction to Machine Learning and Optimization.
Slides: Introduction to Machine Learning and Optimization (same as Set 2 of week 1)
- Week 3,4,5: Neurons, Perceptron Learning, Multi-layer Perceptrons, Backpropagation.
Slides: Multi-layer Perceptrons & Backpropagation
- Week 6: No lecture due to being abroad.
- Week 7: A short intro to artificial and biological vision; Autoencoders.
Slide Set 1: A crash course on artificial and biological vision.
Slide Set 2: Autoencoders; Sparse Autoencoders; k-Sparse Autoencoders; Denoising Autoencoders; Contracting Autoencoders.
- Week 8-9-10: Convolutional Neural Networks.
Slides: Convolution, pooling, non-linearity layers; Filter size, stride, padding parameters; CNN Architectures; Training; Visualization; Applications.
- Week 11-12-13: Recurrent Neural Networks
Slides: Unfolding, Backpropagation Through Time, Vanilla RNN, Long Short-Term Memory, word2vec, Text Modeling, Image Captioning, Machine Translation, Echo State Networks and Resorvoir Computing.
- Week 13: Neural Turing Machines
Slides: Neural Turing Machines, Neural Access Machines, Memory Networks.
- Week 13-14: Boltzmann Machines
Slides: Hopfield Networks, Boltzmann Machines, Restricted Boltzmann Machines, Deep Belief Networks, Deep Boltzmann Machines.
- Week 14: Generative Adversarial Networks, Deep Reinforcement Learning
Slide Set 1: Generative Adversarial Networks (GAN), DC GAN, Mode collapse, Applications with GAN.
Slide Set 2: Reinforcement Learning (RL), RL with deep learning, value networks, policy networks.