tutorial_blogs_videos.org

RNN

• RECURRENT NEURAL NETWORK TUTORIAL, PART 4 – IMPLEMENTING A GRU/LSTM RNN WITH PYTHON AND THEANO

<dd>Note: Intuition behind GRU and LSTM.</dd>

• Understanding LSTM Networks

<dd>Note: a must-read tutorial by Chris Colah</dd>

• Attention and Augmented Recurrent Neural Networks

<dd>Note: Fancy visualization</dd>

CNN

• An Intuitive Explanation of Convolutional Neural Networks
• UNDERSTANDING CONVOLUTIONAL NEURAL NETWORKS FOR NLP

Word vectors

• Distributed Representations

Note: slides from CSC321 by Geoffrey Hinton

Optimization

• Parameter updates

Note: a good overview of variants on SGD (Momentum, AdaGrad, rmsprop).

• Numerical Optimization: Understanding L-BFGS
• Convex Optimization Overview (Part I)
• Convex Optimization Overview (Part II)
• l0-Norm, l1-Norm, l2-Norm, … , l-infinity Norm

Machine Learning

• Introduction to Machine Learning

Note: an excellent video with slides from deep learning summer school 2016

Linear Algebra

• Linear Algebra Review and Reference