Materials

• Lecture slides
• Our videos (russian): lecture, seminar
• Stanford lecture on seq2seq and MT (english) - video
• Alternative CMU lectures - seq2seq and attention

Practice

This time we're gonna use a shared practice.ipynb for both seminar and homework.

Seminar colab url.

RNN Encoder-Decoder (Vanilla):

Attention mechanism:

More on RNN

• Distill.pub post on attention and augmentations for RNN - post
• Seq2seq lecture - video
• BLEU and CIDEr articles.
• Image captioning
  • MSCOCO captioning challenge
  • Captioning baseline notebook
• Lecture on attention mechanisms - video (RUSSIAN)

More on Transformer

• Illustrated transformer post