This is a lecture for the Master in AI for Sustainable Development @ UCL London, a brand new programme that combines the technical side of AI with environmental and humanitarian issues. The topic of the lecture in the Transformer Architecture and its core mechanism, the Self-Attention Mechanism, that is behind many recent breakthrough in Deep Learning.

Here are my contacts

• Twitter (I post about AI/ML/DL and Physics)
• LinkedIn (like twitter, but with a slightly more serious profile picture)
• GitHub repository

Slide

• Theory slides

Notebook

• Coding a Transformer from scratch

Appendix

• Appendix A: Deep Learning Theory and PyTorch Introduction
• Appendix B: Build a Custom Multilayer Perceptron from Scratch

Deep Learning References

Books (must have)

• Deep Learning - Goodfellow, Bengio, Courville