Speech Recognition and Generation course for ITMO AI Talent Hub

• Course materials are available in English in this repo
• All the lectures are given in Russian

Course structure

• 10+ lectures
• 2 personal assignments [50 pts]
• 2 group projects [50 pts]
• 1 research seminar [10 pts]

All courseworks deadlines are provided within their corresponding descriptions

Syllabus

This is a new course that is given for the first time, so the syllabus is subject to slight modifications during the course.

1. Basics of Digital Signal Processing
2. Classic ASR and metrics
3. End-to-End ASR with CTC and audio augmentations
4. Encoder-Decoder End-to-End ASR and decoding with LM
5. Self-supervised speech representations
6. SSL-finetuned ASR and Whisper
7. Text-to-Speech systems
8. Neural vocoders
9. Modern TTS with normalizing flows and diffusion
10. Neural Codec Language Models: VALL-E
11. Extra: Speaker recognition and speech inpainting

Resources

• Deep Learning for Audio (DLA) course
• Edinburgh University: Automatic Speech Recognition (ASR) course
• Stanford CS224S: Spoken Language Processing
• Youtube: Mike X Cohen